Method and system for integrated orthodontic treatment planning using unified workstation

ABSTRACT

A method and workstation for orthodontic treatment planning of a patient. The workstation is based on a computing platform having a graphical user interface, a processor and a computer storage medium containing digitized records pertaining to a patient including image data (3D image data and/or 2D image data). The workstation further includes a set of software instructions providing graphical user interface tools which the user marks a midline and an aesthetic occlusal plane in a two- or three-dimensional virtual model of the patient, marks an occlusal plane in the virtual model; selects a reference tooth in the virtual model; aligns virtual teeth in the virtual model in a proposed arrangement to treat the patient; manages space between the virtual teeth in the proposed arrangement; and repeats one or more of these steps in an iterative fashion to make any further adjustments in the proposed arrangement. When the adjustments are complete, the user selects or identifies a finalized proposed treatment plan for treating the patient.

RELATED APPLICATIONS

This is a continuation-in-part of application Ser. No. 09/834,412, filedApr. 13, 2001, the entire contents of which are incorporated byreference herein.

This application is related to patent applications filed on the samedate as this application, entitled “UNIFIED WORKSTATION FOR VIRTUALCRANIOFACIAL DIAGNOSIS, TREATMENT PLANNING AND THERAPEUTICS”, RohitSachdeva et al, inventors, Ser. No. ______, the entire contents of whichare incorporated by reference herein, and entitled “INTERACTIVE UNIFIEDWORKSTATION FOR BENCHMARKING AND CARE PLANNING”, Rohit Sachdeva et al,inventors, Ser. No. ______, the entire contents of which areincorporated by reference herein.

BACKGROUND OF THE INVENTION

A. Field of the Invention

This invention relates to the field of computerized techniques fororthodontic treatment planning for human patients. More particularly,the invention is directed to an interactive workstation and associatedcomputerized techniques for facilitating integration of various tasksperformed in planning treatment for orthodontic patients.

B. Description of Related Art

The traditional process of diagnosis and treatment planning for apatient with orthodontic problems or disease typically consists of thepractitioner obtaining clinical history, medical history, dentalhistory, and orthodontic history of the patient supplemented by 2Dphotographs, 2D radio graphic images, CT scans, 2D and 3D scannedimages, ultrasonic scanned images, and in general non-invasive andsometimes invasive images, plus video, audio, and a variety ofcommunication records. Additionally, physical models, such as made fromplaster of paris, of the patient's teeth are created from theimpressions taken of the patient's upper and lower jaws. Such models aremanually converted into teeth drawings by projecting teeth on drawingpaper. Thus, there is a large volume of images and data involved in thediagnosis and treatment planning process. Furthermore, the informationmay require conversion from one form to another and selective reductionbefore it could become useful. There are some computerized toolsavailable to aid the practitioner in these data conversion and reductionsteps, for example to convert cephalometric x-rays (i.e., 2 dimensionalx-ray photographs showing a lateral view of the head and jaws, includingteeth) into points of interest with respect to soft tissue, hard tissue,etc., but they are limited in their functionalities and scope. Eventhen, there is a fairly substantial amount of manual work involved inthese steps.

Additionally, a number of measurements, e.g., available space betweenteeth, are also often done manually. Generally, these steps are timeconsuming and prone to inherent inaccuracies. Furthermore, thepractitioner has to contend with the biological interdependencies withinthe patient, which introduces constraints eliminating certain treatmentoptions that would otherwise be acceptable, between the soft tissue, thehard tissue, and the teeth. There is lack of an integrated platformwhich a practitioner could utilize to filter-out non-practicabletreatment options.

Consequently, the practitioner is left to mental visualization, chanceprocess to select the treatment course that would supposedly work.Furthermore, the diagnosis process is some-what ad-hoc and theeffectiveness of the treatment depends heavily upon the practitioner'slevel of experience. Often, due to the complexities of the detailedsteps and the time consuming nature of them, some practitioners take ashort-cut, relying predominantly on their intuition to select atreatment plan. For example, the diagnosis and treatment planning isoften done by the practitioner on a sheet of acetate over the X-rays.All of these factors frequently contribute towards trial and error,hit-and-miss, lengthy and inefficient treatment plans that requirenumerous mid-course adjustments. While at the beginning of treatmentthings generally run well as all teeth start to move at least into theright direction, at the end of treatment a lot of time is lost byadaptations and corrections required due to the fact that the end resulthas not been properly planned at any point of time. By and large, thisapproach lacks reliability, reproducibility and precision. More over,there is no comprehensive way available to a practitioner to stage andsimulate the treatment process in advance of the actual implementationto avoid the often hidden pitfalls. And the patient has no choice anddoes not know that treatment time could be significantly reduced ifproper planning was done.

In recent years, computer-based approaches have been proposed for aidingorthodontists in their practice. However, these approaches are limitedto diagnosis and treatment planning of craniofacial structures,including the straightening of teeth. See Andreiko, U.S. Pat. No.6,015,289; Snow, U.S. Pat. No. 6,068,482; Kopelmann et al., U.S. Pat.No. 6,099,314; Doyle, et al., U.S. Pat. No. 5,879,158; Wu et al., U.S.Pat. No. 5,338,198, and Chisti et al., U.S. Pat. Nos. 5,975,893 and6,227,850, the contents of each of which is incorporated by referenceherein. Also see imaging and diagnostic software and other relatedproducts marketed by Dolphin Imaging, 6641 Independence Avenue, CanogaPark, Calif. 91303-2944.

A method for generation of a 3D model of the dentition from an in-vivoscan of the patient, and interactive computer-based treatment planningfor orthodontic patients, is described in published PCT patentapplication of OraMetrix, Inc., the assignee of this invention,publication no. WO 01/80761, the contents of which are incorporated byreference herein.

Other background references related to capturing three dimensionalmodels of dentition and associated craniofacial structures include S. M.Yamany and A. A. Farag, “A System for Human Jaw Modeling UsingIntra-Oral Images” in Proc. IEEE Eng. Med. Biol. Soc. (EMBS) Conf., Vol.20, Hong Kong, October 1998, pp. 563-566; and M. Yamany, A. A. Farag,David Tasman, A. G. Farman, “A 3-D Reconstruction System for the HumanJaw Using a Sequence of Optical Images,” IEEE Transactions on MedicalImaging, Vol. 19, No. 5, May 2000, pp. 538-547. The contents of thesereferences are incorporated by reference herein.

The technical literature further includes a body of literaturedescribing the creation of 3D models of faces from photographs, andcomputerized facial animation and morphable modeling of faces. See,e.g., Pighin et al., Synthesizing Realistic Facial Expression fromPhotographs, Computer Graphics Proceedings SIGGRAPH '98, pp. 78-94(1998); Pighin et al., Realistic Facial Animation Using Image-based 3DMorphing, Technical Report no. UW-CSE-97-01-03, University of Washington(May 9, 1997); and Blantz et al., A Morphable Model for The Synthesis of3D Faces, Computer Graphics Proceedings SIGGRAPH '99 (August, 1999). Thecontents of these references are incorporated by reference herein.

The present invention is directed to an effective, computer-based,integrated and interactive orthodontic treatment planning system thatprovides the necessary tools to allow the orthodontist to quickly andefficiently design a treatment plan for a patient. The present inventionalso provides a treatment planning system in which theorthodontist-derived parameters for the treatment can be translated intoa design of the treatment. The preferred embodiment integrates 2D and 3Dimages to drive effective treatment planning. Intelligence is built intothe system whereby predefined therapeutic strategies, such asextraction, interproximal reduction, distal movement of molars, can haveassociated value sets predefined by the clinician that are used to drivethe appropriate set-up automatically. Such predefined therapeuticstrategies could be entered via convenient user interface tools, such asby templates.

The treatment design as described herein also allows for real-timecommunication of the treatment plan to occur with the patient, ortransmitted over a communications link and shared with a colleague orremote appliance manufacturing facility. Alternatively, the treatmentplanning can be performed remotely and a digital treatment plan sent tothe orthodontist for review, interactive modification, or approval.

SUMMARY OF THE INVENTION

In a preferred embodiment of the invention, the unified workstation isprovided with software features that facilitate diagnosis and treatmentplanning through a process flow that guides and assists the practitionerin making decisions at various stages of the process in a systematic andcoordinated manner. The workstation includes a computer that stores, andmakes available to the practitioner, records in the form of digital datapertaining to some or all of the following: the patient's clinicalhistory, medical history, dental history, and orthodontic history aswell as 2D photographs, 2D radio graphic images, CT scans, 2D and 3Dscanned images, ultrasonic scanned images, and in general, non-invasiveand optionally invasive images, plus video, audio, and a variety ofcommunication records, such notes, records of office visits, patientletters or communications, etc. All records and images are digitized.The records and images are made available through suitable userinterface icons and graphical displays, which cause display of theimages on the user interface. The images can be combined or superimposedto create a virtual patient model that includes surface features (softtissue) of the patient in one possible embodiment.

The workstation further maintains a comprehensive set of computerinstructions providing tools in the form of icons, screen displays,windows, menus and similar functions and features, accessible throughthe user interface of the workstation to assist the practitioner inplanning the treatment. Various types of tools are contemplated;numerous examples are set forth herein.

From the information gathered, the workstation assists the practitionerin identifying the constraints driven by the practitioner pertinent tothe treatment planning. The treatment planning process flow for apatient typically includes the following steps. To begin with, thegeneral approach to the treatment is postulated or proposed by thepractitioner, based upon attending to the patient's complaints byclinical examination and radiographic images, listening and examinationand, in light of that, assessment of the real problem or problems. Inone possible embodiment, patient information regarding diagnosis andpractitioner-derived constraints are entered into the computer memoryvia the user interface. The supplying of this information could take avariety of forms, including the form of filling in fields of aproblem-oriented matrix, the matrix recording the conditions relevant tothe patient's soft tissue, skeletal, and dental anatomy, each withrespect to vertical, sagittal, and transverse positions.

Next, the workstation provides software tools which enable thepractitioner to mark the facial, dental, maxilla and mandibular midlinesand levels and cant of the upper and lower aesthetic occlusal planes.The designation or marking could be performed on 2D photographs of thepatient displayed on the user interface. In particular, the 2Dphotographs of the patient are recalled from computer memory anddisplayed on the screen display, and the user identifies with the userinterface devices (mouse) these locations, with the assistance ofsuitable user interface tools described herein.

The workstation further is provided with tools for marking the occlusalplane and occlusal plane positions such as upper and lower positionswith respect to posterior, functional, and aesthetic locations on a 2Dlateral X-ray view of the head, jaw and associated anatomicalstructures.

The software provides a feature by which a reference tooth or teeth isidentified. The reference tooth or teeth are a tooth or teeth in whichthe movement of the tooth or teeth is expected to be a minimum over thecourse of treatment. The reference tooth is indicated or marked in anyconvenient fashion, such as by using suitable icons displayedsimultaneously with a x-ray layout of the patient's teeth displayed onthe user interface. Information as to roots of the reference teeth isavailable through X-ray, CT scans, with respect to the crowns of theteeth (2D or 3D).

The workstation further provides software tools for aligning twodimensional image data, such as X-ray data, with a virtualthree-dimensional models of the teeth.

The method and workstation further involves using the graphical userinterface to a) evaluate space between the virtual teeth and arch lengthrequirements, b) evaluate the effect of various constraints on the archlength, and design a desired arch form for the patient, and c) movevirtual teeth in a three-dimensional model of the patient from aninitial orientation relative to the desired arch form, occlusal plane,midline, tooth position, reference tooth so as to arrive at a proposedtooth arrangement for treatment of the patient. Furthermore, theinterdependency of the constraints can be evaluated. Preferably, theuser interface provides tools that enable the practitioner to performthe task of space management between teeth in each arch to assure thatthere would be adequate space to accommodate the contemplated teethalignment. Space management is done with respect to mandible andmaxilla, both at the intra arch and inter arch level.

One important aspect of the invention is that throughout the process,the adjustments made are evaluated against the constraints identified bythe practitioner in the matrix of patient parameters). Potentialviolations between the proposed treatment and the constraints arepreferably pointed out so that only the feasible adjustments can beselected for inclusion in the treatment plan. The constrains areidentified in the marking of midline, occlusal planes, etc. oridentification of the constraints in terms of the shape of the maxillaand mandible, the shape and location of the patient's soft tissue,functional relationships or by other means.

The preferred sequence for these steps is as discussed above; however,the steps can be performed in any order, and repeated as many times asnecessary. The preferred sequence can be driven by the patient's needsor the practitioner's preference. The workstation keeps a record of thelatest changes made at each step. If at any step, the results are notsatisfactory, then one or more of the previous steps might be revisitedand appropriate adjustments made. In this manner the entire process isiterative and closed-loop. These steps described herein can be aprecursor to additional treatment planning and appliance design steps inthe situation where such planning is called for, e.g., in an orthodonticapplication, surgical application, prosthodontic application andrestorative application. When no further adjustments are necessary inany area the treatment plan is considered finalized. This process can beutilized for initial treatment planning or to make treatment adjustmentsbased upon periodic monitoring of patient response to the treatment.

In another embodiment of the invention, assignment of values tovariables, such as markings, tooth alignment, and space allocation, isintegrated in a manner such that the most current selection or value foreach such variable is retained by the workstation and made available atany other step in the treatment planning process besides the one wherethe specific value was originally assigned. For example, the midlinemarking done first on the 2D photos of the patient will be shown in thesame position on appropriate 3D model of the patient's teeth used inteeth alignment and space management. The converse is also true, i.e.,markings on a 3D virtual patient model will be available in 2D images ofthe same patient. An aspect of this invention is that a variable (suchas the location of the midline in a 2D photograph) may be assigned acertain value at one step, and be modified at another step. Suchintegration is realized through proper scaling and synchronization ofpatient's characteristics and features prevalent through 2D photographsand 2D and 3D images and models. A major benefit of this invention isthat throughout the treatment planning process patient's biologicalinterdependence between soft tissue, skeletal, and dental disposition ismaintained and resulting constraints applied in a consistent manner suchthat unfeasible treatment options are filtered out. In turn, the benefitresults in producing efficient and cost effective treatment plans.Additionally it is possible to stage treatment and compare progressagainst the staged treatment. The treatment plan can be actuated orinitiated at any point in treatment to respond to any change that mayoccur.

In yet another aspect of the invention, the unified workstationfacilitates simulation of realizing the target treatment objectivethrough a number of staged incremental or step-wise treatment plans. Thetreatment increments can be varied and its impact seen on associatedteeth disposition and space management. A practitioner can use thisprocess in an interactive and dynamic manner to evaluate severaltreatment scenarios. A major benefit of this invention is that itenables the practitioner and the patient to tailor the treatment planthat best suits the patient needs.

In yet another embodiment of the invention, the unified work stationfacilitates rapid selection of treatment plan driven by templates. Thepractitioner provides specific values or ranges of values for thetreatment parameters, such as for midline, maxilla and mandible levelsand cant for aesthetic occlusal plane, various positions for upper andlower occlusal planes, reference tooth, arch form and alignmentparameters for teeth, and space requirements, etc. for patient. Theunified workstation, using computer instructions based tools, searches aclinical benchmarking knowledge base consisting of reference treatmentplans for a large number of individual patients for a referencetreatment plan. The search essentially cross-references using parametervalues (e.g., archform shape, diagnosis, appliance type, etc.) andsuccessful treatments and finds the most suitable reference treatment,if one is available in the knowledge base. In one possible embodiment,the workstation enables the practitioner to create and update suchknowledge base through a self-learning process aided by computerinstruction tools resident in the unified workstation.

In yet another embodiment of the invention, the unified workstation canbe implemented in a manner such that consultation services from one ormore specialists located remotely from the practitioner can be utilizedon-line by providing capabilities whereby the practitioner and one ormore specialists can all view the same images and charts or otherinformation at the same time and exchange information as appropriate.Thus, the unified workstation can be implemented at a single site or atmultiple sites all working together.

In yet another embodiment of the invention, the unified workstationprovides computer software instruction in the form of user interfacetools that aid in treatment planning in a number of ways. For example,original teeth models can be superimposed on the aligned teeth model,where original teeth positions are shown in one color and the alignedteeth in another color, to visualize the effect of teeth disposition.Constraints (e.g., midline, occlusal plane, etc.) can be defined both in2D and 3D images. They are interchangeable and changes made in 3D areapplied automatically in 2D and vice versa. Occlusal planes can besuperimposed on 3D teeth model to display malocclusion of the originalteeth, and the combination further superimposed on the model of theproperly aligned teeth to see the improvement. The midline can besuperimposed on 3D teeth models. Three-dimensional gingival tissue,obtained from an intra-oral scan, can be superimposed on 3D teeth modelat any stage in the treatment planning process. In a representativeembodiment, color coding is invoked to show different aspects of thedisplay. A grid can be superimposed on 3D teeth model to aid invisualizing certain measurements. Animation light can be thrown on 3Dteeth model, and teeth can be shaded based upon their location withrespect to the animated light source so as to aid in viewing otherwisedifficult to see detailed teeth features. This can also be done while 3Dteeth model is rotated to give views from different angles.

BRIEF DESCRIPTION OF THE DRAWINGS

Presently preferred embodiments of the invention are described below inreference to the appended drawings, wherein like reference numeralsrefer to like elements in the various views, and in which:

FIG. 1 is block diagram of a system for creating a three-dimensionalvirtual patient model and for diagnosis and planning treatment of thepatient.

FIG. 2 is a flow chart showing a method of three-dimensional facecreation from scanning systems, which may be executed in software in thecomputer system of FIG. 1.

FIG. 3 is a flow chart showing an alternative method ofthree-dimensional face model face creation using a plurality of possibleinput image or data formats, which may be executed in software in thecomputer system of FIG. 1.

FIG. 4 is a flow chart showing a method of creating a complete texturedthree-dimensional model of teeth.

FIGS. 4A-4E show a technique for combining 2D color photographs with 3Dtooth data to created textured (colored) 3D tooth models.

FIG. 5 is a screen shot of the user interface of FIG. 1 showing athree-dimensional face model and a three-dimensional tooth model, inseparate coordinate systems (i.e., prior to registration orsuperposition of the two relative to each other). FIG. 5 also shows aplurality of icons, which, when activated, provide tools formanipulating the models shown in the Figure.

FIG. 6 is a screen shot showing one possible method of placement of thelower jaw 3D data into the face data coordinate system usingcorresponding points that are common to each data set.

FIG. 7 is a screen shot showing the face data and the lower jaw 3D datain a common coordinate system (the face coordinate system of FIGS. 5 and6).

FIG. 8 is a screen shot showing the face data and skull data obtainedfrom a CT scan in a common coordinate system.

FIG. 9 is a screen shot showing face data and skull data superimposed onX-ray data obtained from the patient.

FIG. 10 is a screen shot showing the superposition of skull and facedata with X-Ray data.

FIGS. 11A-11E are a series of views of a digital model of an orthodonticpatient obtained, for example from CT scan, photographs, or intra-oralscanning with a hand-held 3D scanner.

FIG. 12 is a diagram illustrating a technique for scaling orthodonticdata obtained from an imaging device, such as a camera, to the actualanatomy of the patient.

FIG. 13 is a diagram showing an alternative scaling method similar tothat shown in FIG. 12.

FIG. 14 is an illustration of an X-ray of a set of teeth and adjacentbone.

FIG. 15 is an illustration of scaling the X-ray data of the tooth to theactual size of the tooth to produce a scaled digital model of the tooth.

FIGS. 16A-16C are an illustration of a method of determining orientationreference points in a digital model of a patient.

FIG. 17 is an illustration of a method of mapping the orientationreference points of FIGS. 16A-16C to a three-dimensional coordinatesystem.

FIG. 18 is an illustration of a method of mapping the orientationreference points of FIGS. 16A-16C to a three-dimensional coordinatesystem.

FIG. 19 is a more detailed block diagram of the treatment planningsoftware executed by the workstation of FIG. 1.

FIG. 20 is an illustration of the integration of the patient dataacquisition, treatment planning and appliance design functions that arefacilitated by a preferred embodiment of the unified workstation.

FIGS. 21-60 illustrate screen shots from the workstation of FIG. 1,showing various treatment planning features that are provided by thetreatment planning software of FIG. 19 in a presently preferredembodiment.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Before describing the treatment planning features of this invention indetail, an overview of a unified workstation will be set forthinitially. The workstation, in a preferred embodiment, provides softwarefeatures that create two dimensional and/or three-dimensional virtualpatient model on a computer, which can be used for purposes of treatmentplanning in accordance with a presently preferred embodiment.

Many of the details and computer user interface tools which apractitioner may use in adjusting tooth position, designing applianceshape and location, managing space between teeth, and arriving at afinish tooth position using interaction with a computer storing anddisplaying a virtual model of teeth are set forth in the priorapplication Ser. No. 09/834,412 filed Apr. 13, 2001, and in publishedOraMetrix patent application WO 01/80761, the contents of which areincorporated by reference herein. Other suites of tools and functionsare possible and within the scope of the invention. Such details willtherefore be omitted from the present discussion.

GENERAL DESCRIPTION

A unified workstation environment and computer system for diagnosis,treatment planning and delivery of therapeutics, especially adapted fortreatment of craniofacial structures, is described below. In onepossible example, the system is particularly useful in diagnosis andplanning treatment of an orthodontic patient. Persons skilled in the artwill understand that the invention, in its broader aspects, isapplicable to other craniofacial disorders or conditions requiringsurgery, prosthodontic treatment, restorative treatment, etc.

A presently preferred embodiment is depicted in FIG. 1. The overallsystem 100 includes a general-purpose computer system 10 having aprocessor (CPU 12) and a user interface 14, including screen display 16,mouse 18 and keyboard 20. The system is useful for planning treatmentfor a patient 34.

The system 100 includes a computer storage medium or memory 22accessible to the general-purpose computer system 10. The memory 22,such as a hard disk memory or attached peripheral devices, stores two ormore sets of digital data representing patient craniofacial imageinformation. These sets include at least a first set of digital data 24representing patient craniofacial image information obtained from afirst imaging device and a second set of digital data 26 representingpatient craniofacial image information obtained from a second imagedevice different from the first image device. The first and second setsof data represent, at least in part, common craniofacial anatomicalstructures of the patient. At least one of the first and second sets ofdigital data normally would include data representing the externalvisual appearance or surface configuration of the face of the patient.

In a representative and non-limiting example of the data sets, the firstdata set 24 could be a set of two dimensional color photographs of theface and head of the patient obtained via a color digital camera 28, andthe second data set is three-dimensional image information of thepatient's teeth, acquired via a suitable scanner 30, such as a hand-heldoptical 3D scanner, or other type of scanner. The memory 22 may alsostore other sets 27 of digital image data, including digitized X-rays,MRI or ultrasound images, CT scanner etc., from other imaging devices36. The other imaging devices need not be located at the location orsite of the workstation system 100. Rather, the imaging of the patient34 with one or other imaging devices 36 could be performed in a remotelylocated clinic or hospital, in which case the image data is obtained bythe workstation 100 over the Internet 37 or some other communicationsmedium, and stored in the memory 22.

The system 100 further includes a set of computer instructions stored ona machine-readable storage medium. The instructions may be stored in thememory 22 accessible to the general-purpose computer system 10. Themachine-readable medium storing the instructions may alternatively be ahard disk memory 32 for the computer system 10, external memory devices,or may be resident on a file server on a network connected to thecomputer system, the details of which are not important. The set ofinstructions, described in more detail below, comprise instructions forcausing the general computer system 10 to perform several functionsrelated to the generation and use of the virtual patient model indiagnostics, therapeutics and treatment planning.

These functions include a function of automatically, and/or with the aidof operator interaction via the user interface 14, superimposing thefirst set 24 of digital data and the second set 26 of digital data so asto provide a composite, combined digital representation of thecraniofacial anatomical structures in a common coordinate system. Thiscomposite, combined digital representation is referred to hereinoccasionally as the “virtual patient model,” shown on the display 16 ofFIG. 1 as a digital model of the patient 34. Preferably, one of the sets24, 26 of data includes photographic image data of the patient's face,teeth and head, obtained with the color digital camera 28. The other setof data could be intra-oral 3D scan data obtained from the hand-heldscanner 30, CT scan data, X-Ray data, MRI, etc. In the example of FIG.1, the hand-held scanner 30 acquires a series of images containing 3Dinformation and this information is used to generate a 3D model in thescanning node 31, in accordance with the teachings of the published PCTapplication of OraMetrix, PCT publication no. WO 01/80761, the contentof which is incorporated by reference herein. Additional data sets arepossible, and may be preferred in most embodiments. For example thevirtual patient model could be created by a superposition of thefollowing data sets: intra-oral scan of the patient's teeth, gums, andassociated tissues, X-Ray, CT scan, intra-oral color photographs of theteeth to add true color (texture) to the 3D teeth models, and colorphotographs of the face, that are combined in the computer to form a 3Dmorphable face model. These data sets are superimposed with each other,with appropriate scaling as necessary to place them in registry witheach other and at the same scale. The resulting representation can bestored as a 3D point cloud representing not only the surface on thepatient but also interior structures, such as tooth roots, bone, andother structures. In one possible embodiment, the hand-held in-vivoscanning device is used which also incorporates a color CCD camera tocapture either individual images or video.

The software instructions further includes a set of functions orroutines that cause the user interface 16 to display the composite,combined digital three-dimensional representation of craniofacialanatomical structures to a user of the system. In a representativeembodiment, computer-aided design (CAD)-type software tools are used todisplay the model to the user and provide the user with tools forviewing and studying the model. Preferably, the model is cable of beingviewed in any orientation. Tools are provided for showing slices orsections through the model at arbitrary, user defined planes.Alternatively, the composite digital representation may be printed outon a printer or otherwise provided to the user in a visual form.

The software instructions further include instructions that, whenexecuted, provide the user with tools on the user interface 14 forvisually studying, on the user interface, the interaction of thecraniofacial anatomical structures and their relationship to theexternal, visual appearance of the patient. For example, the toolsinclude tools for simulating changes in the anatomical position or shapeof the craniofacial anatomical structures, e.g., teeth, jaw, bone orsoft tissue structure, and their effect on the external, visualappearance of the patient. The preferred aspects of the software toolsinclude tools for manipulating various parameters such as the age of thepatient; the position, orientation, color and texture of the teeth;reflectivity and ambient conditions of light and its effect on visualappearance. The elements of the craniofacial and dental complex can beanalyzed quickly in either static format (i.e., no movement of theanatomical structures relative to each other) or in an dynamic format(i.e., during movement of anatomical structures relative to each other,such as chewing, occlusion, growth, etc.). To facilitate such modelingand simulations, teeth may be modeled as independent, individuallymoveable 3 dimensional virtual objects, using the techniques describedin the above-referenced OraMetrix published PCT application, WO01/80761.

The workstation environment provided by this invention provides apowerful system and for purposes of diagnosis, treatment planning anddelivery of therapeutics. For example, the effect of jaw and skullmovement on the patient's face and smile can be studied. Similarly, themodel can be manipulated to arrive at the patient's desired feature andsmile. From this model, and more particularly, from the location andposition of individual anatomical structures (e.g., individual toothpositions and orientation, shape of arch and position of upper and lowerarches relative to each other), it is possible to automatically backsolve for or derive the jaw, tooth, bone and/or soft tissue correctionsthat must be applied to the patient's initial, pre-treatment position toprovide the desired result. This leads directly to a patient treatmentplan.

These simulation tools, in a preferred embodiment, compriseuser-friendly and intuitive icons 35 that are activated by a mouse orkeyboard on the user interface of the computer system 10. When theseicons are activated, the software instruction provide pop-up, menu, orother types screens that enable a user to navigate through particulartasks to highlight and select individual anatomical features, changetheir positions relative to other structures, and simulate movement ofthe jaws (chewing or occlusion). Examples of the types of navigationaltools, icons and treatment planning tools for a computer user interfacethat may be useful in this process and provide a point of departure forfurther types of displays useful in this invention are described in thepatent application of Rudger Rubbert et al., Ser. No. 09/835,039 filedApr. 13, 2001, the contents of which are incorporated by referenceherein.

The virtual patient model, or some portion thereof, such as datadescribing a three-dimensional model of the teeth in initial and targetor treatment positions, is useful information for generating customizedorthodontic appliances for treatment of the patient. The position of theteeth in the initial and desired positions can be used to generate a setof customized brackets, and customized flat planar archwire, andcustomized bracket placement jigs, as described in the above-referencedAndreiko et al. patents. Alternatively, the initial and final toothpositions can be used to derive data sets representing intermediatetooth positions, which are used to fabricate transparent aligning shellsfor moving teeth to the final position, as described in theabove-referenced Chisti et al. patents. The data can also be used toplace brackets and design a customized archwire as described in thepreviously cited application Ser. No. 09/835,039.

To facilitate sharing of the virtual patient model among specialists anddevice manufacturers, the system 100 includes software routines andappropriate hardware devices for transmitting the virtual patient modelor some subset thereof over a computer network. The system's softwareinstructions are preferably integrated with a patient management programhaving a scheduling feature for scheduling appointments for the patient.The patient management program provides a flexible scheduling of patientappointments based on progress of treatment of the craniofacialanatomical structures. The progress of treatment can be quantified. Theprogress of treatment can be monitored by periodically obtaining updatedthree-dimensional information regarding the progress of treatment of thecraniofacial features of the patient, such as by obtaining updated scansof the patient and comparison of the resulting 3D model with theoriginal 3D model of the patient prior to initiation of treatment.

Thus, it is contemplated that system described herein provides a set oftools and data acquisition and processing subsystems that togetherprovides a flexible, open platform or portal to a variety of possibletherapies and treatment modalities, depending on the preference of thepatient and the practitioner. For example, a practitioner viewing themodel and using the treatment planning tools may determine that apatient may benefit from a combination of customized orthodonticbrackets and wires and removable aligning devices. Data from the virtualpatient models is provided to diverse manufacturers for coordinatedpreparation of customized appliances. Moreover, the virtual patientmodel and powerful tools described herein provide a means by which thecomplete picture of the patient can be shared with other specialists(e.g., dentists, maxilla-facial or oral surgeons, cosmetic surgeons,other orthodontists) greatly enhancing the ability of diversespecialists to coordinate and apply a diverse range of treatments toachieve a desired outcome for the patient. In particular, the overlay orsuperposition of a variety of image information, including 2D X-Ray, 3Dteeth image data, photographic data, CT scan data, and other data, andthe ability to toggle back and forth between these views and simulatechanges in position or shape of craniofacial structures, and the abilityto share this virtual patient model across existing computer networks toother specialists and device manufacturers, allows the entire treatmentof the patient to be simulated and modeled in a computer. Furthermore,the expected results can be displayed before hand to the patient andchanges made depending on the patient input.

With the above general description in mind, additional details ofpresently preferred components and aspects of the inventive system andthe software modules providing the functions referenced above will bedescribed next.

Capture of Image Information

The creation of the virtual patient model uses the capture and storageof at least two different digital sets of image data of the patient. Theimage sets will typically represent, at least in part, overlappingcraniofacial anatomical structures so that a superposition of them in acommon three-dimensional coordinate system may occur.

The type of image data that will be obtained will vary depending on theavailable image acquisition devices available to the practitioner.Preferably, the system employs software simulation of changes in shapeor position of craniofacial structures (e.g., teeth or jaw) on thevisual appearance, e.g., smile, of the patient. Accordingly, at leastone of the data sets will include normally include data regarding thesurface configuration of the face and head. A commercially availabledigital CCD camera 28 (FIG. 1), e.g., camera available from Sony orCanon, can be used to obtain this information. Preferably, the imagedata is color image data. The data sets are obtained by photographingthe patient's head and face at various viewing angles with the cameraand storing the resulting image files in the memory of the computer.These images can provide a basis for creating a morphable face model.

The image data regarding the patient's exterior appearance can beobtained through other means including via scanning of the head and faceof the patient via the hand-held 3D-scanner 30 described in thepublished OraMetrix PCT application, referenced previously. If thisapproach is used, it may be beneficial to apply a thin layer ofnon-toxic, opaque and reflective substance to the skin prior to scanningto insure adequate data capture by the hand-held scanner. A suitableopaquing substance is described in the patent application of NancyButcher et al. Ser. No. 10/099,042 filed Mar. 14, 2002, entitled “Methodfor Wet-Field Scanning,” the contents of which are incorporated byreference herein. In operation, the scanner captures a sequence ofoverlapping images of the surface of the patient as the scanner is heldby the hand and moved about the face. The set of images can be obtainedin only a few minutes. Each image is converted to a set of X, Y and Zcoordinate positions comprising a cloud of points representing thesurface of the face. The point clouds from each image are registered toeach other to find a best fit to the data. The resulting registeredpoint cloud is then stored in the memory as a virtual three-dimensionalobject. The construction, calibration and operation of the scanner, andthe manner of converting scanned data to point clouds and registeringthree-dimensional point clouds to form a three-dimensional object isdescribed at length in the published PCT application of OraMetrix andtherefore omitted from the present discussion for the sake of brevity.Other types of scanners or coordinate measuring instruments could beused in less preferred embodiments, such as the scanning devices in theYamany et al. articles referenced previously.

Aside from surface data of the patient obtained by the camera 28 or 3Dscanner 30, the system typically will include the capture of additionaldata representing the teeth of the patient, and also capture ofadditional data representing craniofacial structures not visible to thenaked eye using other imaging devices 36 (FIG. 1). For example, thesystem will acquire digitized images from an X-ray machine capturingX-ray photographs of the patient's head, jaw, teeth, roots of teeth, andother craniofacial structures. These photographs are digitized andstored in the memory of the computer system.

As other possible examples, three-dimensional magnetic resonance imagesof the patient's head or jaws are obtained and stored in the memory.Other examples include images acquired from a computed tomography (CT)scanner, ultrasound imager, or other type of imaging device.

While the above discussion has described how 3D image of the face can beobtained from a three-dimensional scanner, there are other possibilitiesthat may be used in the practice of alternative embodiments. One suchalternative is creating a 3D virtual face from a series of 2-D colorphotographs. This technique is known and described in Pighin et al.,Synthesizing Realistic Facial Expression from Photographs, ComputerGraphics Proceedings SIGGRAPH '98, pp. 78-94 (1998); Pighin et al.,Realistic Facial Animation Using Image-based 3D Morphing, TechnicalReport no. UW-CSE-97-01-03, University of Washington (May 9, 1997); andBlantz et al., A Morphable Model for The Synthesis of 3D Faces, ComputerGraphics Proceedings SIGGRAPH '99 (August, 1999), the contents of whichare incorporated by reference herein. Basically, in this alternative,two-dimensional color pictures of the face are taken which are convertedautomatically to a textured 3 dimensional model using a ‘morphablemodel’ technique. Here, the phrase “textured 3 dimensional model” isused in the particular sense of a colorized three-dimensional object,with the word “texture” synonymous with color data, as that term is usedin this particular art.

Morphable models can be built based on various known approaches such asoptic flow algorithms or active model matching strategy, or acombination of both. One approach is to scan a set of 2D faces. A shapevector containing 3D vertices and texture vector containing RGB colorvalues of each vertex represents the geometry of the face. Each face isdivided into sub regions such as eyes, nose, mouth etc. Blending thesub-regions at the borders generates the complete 3D face. Automaticmatching and estimating 3D face of a 2D color image from morphable modelis carried out as follows:

New Shape (Sn) and texture (Tn) are computed as follows:

Sn=Sa+Σαs;  (1)

Tn=Ta+Σβt,  (2)

where Sa and Ta are the averages of Shape S and Texture T over all the3D face datasets; s & t are the eigenvectors of the covariance matrices;α and β are the coefficients of the facial shape and texture for all thefaces, and n is a sub-region index.

Rendering parameters ρ contain camera position, object scale, imageplane rotation and translation and light intensity. From Bayes decisiontheory, the set of parameters, (α,β,ρ) are determined with maximumposterior probability for getting a corresponding 3D face from a 2Dimage.

Three-dimensional image data sets of the upper and lower archesincluding upper and lower teeth are preferably created with a 3D opticalscanner 30, such as the OraMetrix hand-held in-vivo scanner. If the 3Djaw model has no texture model, i.e., no color data, the texture datacan be extracted from the 2 dimensional colored picture of the upper andlower jaw and mapped to the 3D coordinates on the jaw model using acylindrical projection technique. In this technique, a map isconstructed in texture space, that for each point (u, v), specifies atriangle whose cylindrical projection covers that point. The 3D point pcorresponding to point (u, v) in texture space is computed byintersecting a ray with the surface of the corresponding point in the 2Dcolored image.

Superposition or Registration of the Data Sets

After the images of the face, craniofacial structures, X-rays, teethetc. are obtained and stored in memory in digital form they aresuperimposed on each other (i.e., registered to each other via softwarein the workstation) to create a complete virtual patient model on theworkstation. The superposition of the sets of image data may bedeveloped as an automatic software process, or one in which there isuser involvement to aid in the process. In one possible example, thethree-dimensional textured model of the face is properly aligned withthe 3D jaw model obtained from the intra-oral scan, 3D skull data fromCT scan, and 2 dimensional X-rays to create a virtual patient model. Forcorrect alignment of the data sets to each other, a preferred methodexecuted by the software selects three or more corresponding points onthe 3D jaw and the 3D face, and then computes a transformation matrix tore-orient the 3D face relative to the 3D jaw. This transformation matrixwill contain the information needed to rotate and translate the 3D facerelative to the 3D jaw in a best-fit manner to align the two to eachother. Methods of calculation of transformation matrices to achieveregistration are taught in the published PCT patent application ofOraMetrix, Inc., cited previously. Similar methods are used forregistering the CT scan data and X-ray data to the combined 3D face andjaw model. Once the superposition is achieved, the resulting model isdisplayed on the workstation user interface. The user is provided withtools for simulating movement or repositioning of craniofacialstructures of the virtual patient, and the computer animates suchmovement or repositioning and shows the effect of such movement orrepositioning on the external visual appearance of the patient.

An example of registering scan data of a jaw from an intra-oral scan toa 3D face model using human interaction is shown in FIGS. 2-7. FIG. 2 isa flow chart showing a software method of three-dimensional facecreation from scanning systems, which may be executed in software in thecomputer system 10 of FIG. 1. There are two possible approaches forcreating the 3D face, one using a color digital camera 28 (FIG. 1) andanother using scanning of the face using the hand held scanner 30 andscanning node 31 (FIG. 1). In the situation in which a color digitalcamera is used, at step 40 a set 24 of 2D digital color photographicimages of the face and head are obtained and stored in the memory 22 ofFIG. 1. The set 24 of images is supplied to a module 42 which creates avirtual 3D face using an active model matching strategy, using thetechniques known in the art and described in Pighin et al., SynthesizingRealistic Facial Expression from Photographs, Computer GraphicsProceedings SIGGRAPH '98, pp. 78-94 (1998); Pighin et al., RealisticFacial Animation Using Image-based 3D Morphing, Technical Report no.UW-CSE-97-01-03, University of Washington (May 9, 1997); and Blantz etal., A Morphable Model for The Synthesis of 3D Faces, Computer GraphicsProceedings SIGGRAPH '99 (August, 1999). This 3D face model is thenstored in the hard disk memory of the computer 10, as indicated atprocess module 44.

In alternative embodiments, a 3D scanning of the face using a laser or3D optical scanner is performed, as indicated at 44. The 3D model isprovided to a module 46 which creates a morphable model of the face andhead with an optic flow algorithm. This morphable model is provided tothe module 42 for creating a 3D face. At step 50, the software inquiresas to whether a morphable 3D face is available, and if not theprocessing of module 42 executes, in which a 3D morphable model of theface is created. If a morphable 3D face is already available, thesoftware inquires at step 54 as to whether texture (color) informationis available to add to the 3D face. (Note that in many 3D scannersystems there is no acquisition of color information, only spatialinformation). If color information is not available, the processingproceeds to module 56. In module 56, the color data is provided to the3D model to create a 3D color morphable virtual model. The color data issupplied from the digital photographs of the patient, obtained at step40. The texture information is supplied to the 3D model from the scannerusing a cylindrical projection technique in module 56 (or by using anyother known technique). The textured, morphable 3D model of the face andhead is stored as indicated at module 44.

An alternative software method or process for creating a 3D model of theface is shown in FIG. 3. The method involves the acquisition of a 3Dcolor face model 52 (using for example the techniques of FIG. 2), theacquisition of 3D color model of the teeth 54, and the acquisition of amodel 56 of the skull using a CT scanner. These three models aresupplied to a module 58 which performs an aligning transformation on thedata sets from each of these modules. The aligning transformationprocess 58 basically scales and provides the necessary X, Y and Ztranslations and rotations to place the data sets into a commoncoordinate system such that common anatomical structures overlap eachother. The complete 3D face model is stored as indicated at step 60 andthen supplied to an overlay transformation module 66. The overlaytransformation module 66 obtains a set of 2D color face photographs 62and X-Rays 64, and overlays them to the complete 3D face model to resultin a combined, composite model of the face, skull, teeth, and associatedtooth roots, bone and other anatomical data. This compositerepresentation of the patient is stored in a database 68 for the system100.

FIG. 4 shows a process that can be used to combine 3D scan data with 2Dcolor photographs to create a 3D color model of the teeth. In step 70,the teeth are scanned with the hand-held intra-oral scanner 30 ofFIG. 1. The resulting data represent a 3D model of the dentition, whichis stored in the computer 10. This process is described in the publishedPCT application of OraMetrix, Inc. cited previously. At step 72, 2Dcolor photographs of the teeth are obtained with a color digital camera.In one possible embodiment, the hand-held scanner 30 includes a videocamera that obtains a continuous stream of color video frames separateand apart from the acquisition of 3D image data. The color photographsof the dentition at step 72 could be obtained in this manner.

At step 74, a 3D textured model of the teeth is created using acylindrical projection technique. Basically, in this technique, thecolor data from the color photographs is projected onto the tooth data.The tooth data can be represented as triangular surfaces, with thevertices of each triangle being adjacent points in a point clouddefining the surface of the tooth. The color is projected on thesurfaces, and each surface is assigned a value associated with aparticular color. The result is a 3D color model of the teeth.

FIGS. 4A-4E show several screen displays from a user interface of theunified workstation that illustrate the process of texture mapping a 3Dobject (here, teeth) by projection of color data from a 2D photograph.After a patient's dentition is scanned, the virtual teeth and gingivafor both upper and lower arches are represented as a single surface, inthe present example a triangle mesh surface. FIG. 4A shows a 2D digitalphotograph of teeth/gingivae 71 displayed in a graphical window 73 alongwith a 3D virtual model of the teeth 75 to one side. The 2D digitalphotograph 71 is scaled up or down in size as necessary to as to beapproximately the same in scale (size) as the 3D model of the teeth 75.This is accomplished using any suitable icons or mouse action, such asclicking on the 2D photograph and scrolling up or down with the mouse tochange the size of the 2D image so that it matches the size of the 3Dmodel. FIG. 4B shows the surface of the teeth and gingivae of the 3Dvirtual model 75 in greater detail. The surface of the model 75comprises a set of minute interconnecting triangle surfaces, with thevertices of the triangle surfaces being points that represent thesurface. This is only one possible format for representing the surfaceof a 3D object.

After the 2D photograph and 3D model have been scaled, a translation isperformed to as to overlap the 3D model and the 2D photograph. FIG. 4Cshows the 2D picture 71 transformed by scaling and translation such thatit is superimposed on the 3D model 75. This superposition could beperformed manually or automatically. For example, the user can click anddrag the 2D digital photograph 71 and manually move it using the mouseso that it overlaps exactly the 3D model 75. The color information inthe 2D photograph 71 is projected and mapped to the individual trianglesurfaces forming the lower jaw and upper jaw of the 3D model 75 using,for example, a projection algorithm. The result, a textured 3D model, isshown in FIG. 4D. FIG. 4E shows textured 3D model after rotation on theuser interface.

Occlusal and lingual 2-D color photographs of each jaw are also obtainedand texture data is mapped to the surface data. The result is a completetrue color 3D model of the teeth of both arches.

FIG. 5 is an illustration of a screen display on the user interface ofthe computer 10. The display shows a 3D morphable model 102 of patienton the left hand side of the display, in a given arbitrary coordinatesystem X, Y, Z. The morphable model 102 is obtained, for example, fromcolor photographs using the techniques described previously. Athree-dimensional model 104 of teeth of the patient is shown on theright hand side of the screen. The 3D model of the teeth 104 can beobtained from intra-oral scanning using the scanner 30 of FIG. 1, from alaser scan of a physical model of the dentition obtained from animpression, from a coordinate measuring device or some other source. Thesource is not particularly important. The 3D model of the teeth 104 isshown in a separate coordinate system X′, Y′, Z′. Screen displayincludes various icons 35 the allow the user to position the tooth model104 relative to the morphable model 102 in order to combine the two in acommon coordinate system and construct a composite model.

In FIG. 6, the user has activated an “Align References” icon, whichcauses the screen display to show the box 106 on the left hand side ofthe screen. The user is provided with the option to pick points thatrepresent anatomical structures that are common to both the morphablemodel 102 and the 3D tooth model 104. In this particular situation, theuser has selected with the mouse two points on the lower arches whichlie at the intersection of the teeth and the gums. These two points areshown as a triangle 108 and a square 110. Obviously, other points couldbe chosen. The user then clicks on the “Apply” tab 112. The result isshown in FIG. 7, in which the 3D tooth model 104 is combined with themorphable face 102 model to produce a combined virtual patient model 34.

In the example of FIGS. 5-7, the morphable model 102 was already scaledto the same scale as the tooth model 104. In other words, the datarepresenting the morphable face model indicates that the spatialdimensions of the teeth in the morphable face model is substantially thesame as the spatial dimensions of the virtual tooth model 104. Methodsof performing scaling are described below.

FIG. 8 is an illustration of an alternative embodiment of a virtualpatient model 34. In this embodiment, the model 34 is a combination ofdata 102 representing a morphable face, obtained from a plurality of 2Dcolor photographs, and skull data 114 obtained from a CT scan. The twosets of data are shown in a common coordinate system, appropriatelyscaled. The manner of combining the two data sets can be using theapproach described in FIGS. 6 and 7. Alternatively, the user could clickand drag using a mouse one or the other of the data sets and manuallyposition it until it is in the correct position. As yet anotheralternative, the software could be programmed to find common overlappingfeatures (such as for example teeth) using surface matching algorithms,and then position the CT scan model relative to the face model such thatthe common features overlap exactly.

FIG. 9 is a screen shot of yet another possible embodiment of a virtualpatient model. This model combines face data 102 from a morphable facemodel (obtained from 2D color photographs), skull data 114 from a CTscan, and X-ray data 116 obtained from a set of digital X-Rays of thepatient. The manner of creating the virtual patient model can be forexample using the procedure of FIG. 3 and FIG. 6-7. The morphable facemodel is aligned relative to the CT scan data either automatically orusing some human interaction. The 2D X-Ray data can be morphed into 3Ddigital data using the morphable model algorithms cited previously.Alternatively, the 2D X-Ray data can be combined with 3D optical scandata of crowns of the teeth to create a combined X-Ray/3D tooth model,which is then combined with the CT/morphable face model. This processmay be optimized by using virtual template tooth roots, which aremodified to fit the X-ray data, and then this combined 3D root model iscombined with crown data to produce a complete set of 3D teeth,including roots. This combined model is merged into the CTscan/morphable face model using the techniques of FIGS. 6 and 7(selecting common points then using the “Apply Points” icon, FIG. 7,item 112), using click and drag techniques, or any other appropriateregistration or transformation technique.

Once the virtual model is created, the user is provided with tools thatallow the user to hide one or more image data in order to study certainfeatures. Furthermore, the user is provided with navigation tools withthe ability to manipulate the model so as to view it from anyuser-specified perspective. For example, in FIG. 10 a screen shot isshown of the superposition of skull data 114 with X-Ray data 116. Inthis example, complete 3D models of the teeth 116 are created from X-raydata using the algorithms described previously. Alternatively, thecomplete 3D tooth models 116 are created from combining X-Ray data with3D models of teeth obtained by a scan of the crowns of the teeth (usingthe scanner 30 of FIG. 1 or from a laser scan of a physical model of thedentition), and/or with the use of template tooth roots that aremodified to match the X-ray data.

Scaling of Data

When digital image data from multiple sources are combined orsuperimposed relative to each other to create a composite model, it maybe necessary to scale data from one set to the other in order to createa single composite model in a single coordinate system in which theanatomical data from both sets have the same dimensions inthree-dimensional space. Hence, some scaling may be required. Thissection describes some approaches to scaling that may be performed inone possible embodiment of the invention.

FIG. 11A-11E are views of scan data 200 representing the front, rightside, left side, lower and upper arches of a patient. The data includesthe teeth 202 and the gingiva 204. FIG. 12 illustrates a technique ofscaling the orthodontic data to match the actual orthodontic size.Depending on of the scanning technique, the orthodontic data will notcompletely reproduce the exact size of the teeth and other portions ofthe orthodontic structure. To facilitate the accuracy of thethree-dimensional digital model, at least one tooth 206 can be markedutilizing one or more markings 208. The marking is done prior toobtaining the orthodontic data. Once the orthodontic data for the tooth206 is obtained, scaling reference points 210 are also obtained. Thescaling reference points are the points in the scan data that representthe image of the markings 208. A determination between the differencesbetween the scaling reference points 210 and the actual markings 208determine a scaling factor 212. As one of average skill in the art willreadily appreciate, having the actual markings 208 and the scalingreference points 210, a variety of mathematical operations may be usedto determine the scaling factor 212. For example, the coordinatedifferences (distance) between each of the vertices of the triangle maybe utilized. As one of average skill in the art will further appreciate,a different number of markings 208 may be utilized. For example, twomarkings may be used or four markings may be used, etc. In addition,more than one tooth may be marked with similar markings 208. Note thatthe markings may be on the exterior of the patient, and a localtriangulation technique may be used to obtain the scaling factor.Further note that the scaling factor 212 determination is based on anassumption that the scan data will have a linear error term in each ofthe x, y and z axis, such that a single scaling factor is determined andused to scale each of the teeth as well as the other aspects of theorthodontic structure of the patient.

FIG. 13 illustrates an alternate marking technique for determining ascaling factor for the orthodontic data. As shown, an actual tooth 206is marked with a marking 208. The marking 34 is of a substantial size soas to be adequately measured. Once the orthodontic data is obtained, theorthodontic data of the tooth 202 and a corresponding scaling referencepoint (area) 210 are used to determine the scaling factor 212. As one ofaverage skill in the art will readily appreciate, a simple mathematicalfunction may be used to determine the scaling factor 212 based on thesize (diameter) difference between the actual marking 34 and the scalingreference point 36. As an alternative to marking as described withreference to FIGS. 12 and 13, the actual tooth size, and the size of themodel of the tooth, may be measured and used to determine the scalingfactor. Accordingly, the difference between the actual tooth size thesize of the tooth in the scan data will constitute the scaling factor.

When three-dimensional scanning of the type described in the publishedPCT application or OraMetrix is used, scaling of the three-dimensionaldata is not needed as a true, accurate and to scale three-dimensionalimage is obtained through the use of triangulation. Likewise, a truethree-dimensional image can be obtained techniques such as computedtomography. However, for video or photographic data, and for X-ray data,scaling such as shown in FIGS. 12 and 13 may be needed in order to mergethat data to other data such as 3D scan data.

FIG. 14 illustrates a two-dimensional representation of image data, suchas a graphical diagram of a radiographic image, such as an x-ray of afew teeth. In another embodiment, the radiographic image can be acomputed tomographic image volume. As previously mentioned, theorthodontic data contains three-dimensional images of the surface of theorthodontic structure. X-rays provide a more detailed view of the teethand surrounding hard and soft tissue as two dimensional image data. Asshown in FIG. 14, each tooth 206 includes a crown 220, and a root 222and is embedded in bone 224. Accordingly, the orthodontic data 200 ofFIG. 11 only illustrates the crown 206 of the teeth. As such, a completethree-dimensional model of the orthodontic patient requires the rootsand bone to be included.

FIG. 15 illustrates a technique of using the scaled digital model 226 ofthe tooth's crown to produce an integrated or composite digital model228 of the tooth. In this embodiment, the x-rayed data 230 of the toothis used in comparison with the scaled digital model 226 to determine aper tooth scaling factor. The scaled digital model 226 of the tooth ispositioned to be planar with the x-ray of the tooth 230. Having obtainedthe proper orientation between the two objects, the per tooth scalingfactor is determined and subsequently used to generate the compositescaled digital model 228 of the tooth. In a specific embodiment, the pertooth scaling factor is required for current x-ray technologies, sincex-rays produce a varying amount of distortion from tooth to toothdepending on the distance of the tooth from the film, the angle of x-raytransmission, etc.

To more accurately map the two-dimensional images of a tooth onto thethree-dimensional model, multiple angles of the tooth should be used.Accordingly, a side, a front, and a bottom view of the tooth should betaken and mapped to the scaled digital model of the tooth. Note that thebone and other portions of the orthodontic structure are scaled in asimilar manner. Further note that MRI images, and any other imagesobtained of the orthodontic patient, may also be scaled in a similarmanner. A more complete representation of the tooth roots may beobtained using standardized, template 3D virtual tooth roots, applyingthe X-Ray data to the template tooth roots and modifying their shapeaccordingly, and them applying the modified template tooth root to thescan data of the crown to create a scaled, complete virtual tooth objectincluding tooth roots.

FIGS. 16A-16C illustrate a graphical diagram of selecting orientationreference points based on physical attributes of the orthodonticstructure. The orientation reference points 262 and 266 will besubsequently used to map the digital image of the orthodontic structureinto a three-dimensional coordinate system that will not change duringthe course of treatment. In this example, the frenum 264 has beenselected to be one of the orientation reference points 266 and the rugae260 has been selected as the other reference point 262. The frenum 264is a fixed point in the orthodontic patient that will not change, orchange minimally, during the course of treatment. As shown, the frenum264 is a triangular shaped tissue in the upper-portion of the gum of theupper-arch. The rugae 260 is a cavity in the roof of the mouth 268 inthe upper-arch. The rugae will also not change its physical positionthrough treatment. As such, the frenum 264 and the rugae 260 are fixedphysical points in the orthodontic patient that will not change duringtreatment. As such, by utilizing these as the orientation referencepoints 262 and 266, a three-dimensional coordinate system may be mappedthereto. Note that other physical attributes of the orthodontic patientmay be used as the orientation reference points 262 and 266. However,such physical points need to remain constant throughout treatment.Accordingly, alternate physical points include the incisive papilla,cupid's bow, the inter-pupillar midpoint, inter-comissural midpoint(e.g., between the lips), inter-alar midpoint (e.g., between the sidesof the nose), the prone nasale (e.g., the tip of the nose), sub-nasale(e.g., junction of the nose and the lip), a dental mid-line point, apoint on the bone, a fixed bone marker such as an implant (e.g., a screwfrom a root canal, oral surgery).

The x, y, z coordinate system may be mapped to the physical points onthe digital model of the orthodontic structure in a variety of ways. Inone example, the origin of the x, y, z coordinate system may be placedat the frenum 264, the z-axis aligned with reference to the frenum andthe rugae 260, and the x-axis is aligned with the midline of the upperand/or lower arch. This is further illustrated in FIGS. 17 and 18. Notethat an external positioning system may be used to obtain theorientation reference points. For example, the patient may sit in achair at a specific location of an examination room that includes atriangulation positioning system therein. As such, when the patient isscanned, the scanned images may be referenced with respect to the room'striangulation positioning system.

FIG. 17 illustrates a graphical representation of mapping theorientation reference points 262 and 266 to the x-z plane of thethree-dimensional x, y, z coordinate system. In this illustration,orientation point 266, which corresponds to the frenum 264, is selectedas the origin of the x, y, z coordinate system. Note that any locationmay be selected as the origin 72. The orientation points 262 and 266 areused to determine an x, z plane orientation angle 262. Typically, the x,y, z coordinate system will be selected such that when looking at thepatient from a frontal view, the x direction will be to right of thepatient, the y direction towards the top of the patient's head and the zdirection will be out away from the patient. As one of average skill inthe art will appreciate, the orientation of the x, y, z plane may be inany orientation with respect to the reference points 262 and 266.

The x-y plane is mapped to the orientation reference point 262 and 266as shown in FIG. 18. The orientation reference point 262 and 266 areused to generate an x-y plane orientation angle 284. Based on the x-yplane orientation angle 284 and the x-z plane orientation angle 262, adigital model of a tooth 270 may be positioned in three-dimensionalspace with respect to the x, y, z coordinate system. As shown in FIGS.17 and 18, the digital model of the tooth 270 includes a tooth depth278, an angle of rotation 276 with respect to the x-z axis, an angle ofrotation 282 with respect to the x-y plane, a positioning vector 274which is in a three-dimensional space, the length of the tooth includingthe crown dimension, and the root dimension. Accordingly, each tooth isthen mapped into the x, y, z coordinate system based on the tooth'scenter, or any other point of the tooth, and the dimensions of thedigital model of the corresponding tooth. Once each tooth has beenplaced into the x, y, z coordinate system, the digital model of thetooth is complete. Note that the lower-arch is also referenced to the x,y, z coordinate system wherein the determination is made based on theocclusal plane of the patient's orthodontic structure. Alternatively,the lower-arch may include a separate three-dimensional coordinatesystem that is mapped to the coordinate system of the upper-arch. Inthis latter example, fixed points within the lower-arch would need to bedetermined to produce the lower arch's three-dimensional coordinatesystem.

Treatment Planning

The computer or workstation 10 (FIG. 1) that includes the software forgenerating the patient model preferably includes interactive treatmentplanning software that allows the user to simulate various possibletreatments for the patient on the workstation and visualize the resultsof proposed treatments on the user interface by seeing their effect onthe visual appearance of the patient, especially their smile. Theinteractive treatment planning preferably provides suitable tools andicons that allow the user to vary parameters affecting the patient. Suchparameters would include parameters that can be changed so as tosimulate change in the age of the patient, and parameters that allow theuser to adjust the color, texture, position and orientation of theteeth, individually and as a group. The user manipulates the tools forthese parameters and thereby generates various virtual patient modelswith different features and smiles. The patient models are displayed onthe user interface of the workstation where they can be shared with thepatient directly. Alternatively, the workstation can be coupled to acolor printer. The user would simply print out hard copies of the screenshots showing the virtual patient model.

The manner in which the software is written to provide tools allowingfor simulation of various parameters can vary widely and is notconsidered especially critical. One possibility is a Windows-basedsystem in which a series of icons are displayed, each icon associatedwith a parameter. The user clicks on the icon, and a set of windows aredisplayed allowing the user to enter new information directing a changein some aspect of the model. The tools could also include slide bars, orother features that are displayed to the user and tied to specificfeatures of the patient's anatomy. Treatment planning icons for movingteeth are disclosed in the published PCT application of OraMetrix, Inc.,WO 01/80761, which gives some idea of the types of icons and graphicaluser interface tools that could be used directly or adapted to simulatevarious parameters.

Once the user has modified the virtual patient model to achieve thepatient's desired feature and smile, it is possible to automaticallyback-solve for the teeth, jaw and skull movement or correction necessaryto achieve this result. In particular, the tooth movement necessary canbe determined by isolating the teeth in the virtual patient model,treating this tooth finish position as the final position in theinteractive treatment planning described in the published OraMetrix PCTapplication, WO 01/80761, designing the bracket placement and virtualarch wire necessary to move teeth to that position, and then fabricatingthe wire and bracket placement trays, templates or jigs to correctlyplace the brackets at the desired location. The desired jaw movement canbe determined by comparing the jaw position in the virtual patientmodel's finish position with the jaw position in the virtual patientmodel in the original condition, and using various implant devices orsurgical techniques to change the shape or position of the jaw toachieve the desired position.

The virtual patient model as described herein provides a common set ofdata that is useable in a variety of orthodontic or other treatmentregimes. For example, the initial and final (target) digital data setsof the patient's tooth positions can be relayed to a manufacturer ofcustomized transparent removable aligning shells for manufacture of aseries of aligning devices, as taught in the Chisti et al. patents citedpreviously. Alternatively, the tooth positions may be used to derivecustomized bracket prescriptions for use with a flat planar archwire.

The choice of which treatment modality, and whether to use anyadditional treatment or therapeutic approaches (including surgery) willdepend on the patient in consultation with the treating physician. Theintegrated environment proposed herein provides essentially a platformfor a variety of possible treatment regimes. Further, the creation anddisplay of the virtual patient model provides for new opportunities inpatient diagnosis and sharing of patient information across multiplespecialties in real time over communications networks.

FIG. 19 is a block diagram of an integrated workstation environment forcreation of the virtual patient model and diagnosis, treatment planningand delivery of therapeutics. The workstation environment shown in blockdiagram form in FIG. 19 may incorporate many of the hardware aspectsshown in FIG. 1, including scanning or imaging devices 28/36 forcapturing two dimensional images, such as a color digital camera orX-Ray machine. The workstation environment will preferably includescanning or imaging devices 30/36 for capturing three dimensional imagesand creating 3D models of the patient, including one or more of thefollowing: laser scanners for scanning a plaster model of the teeth,optical scanner such as the OraMetrix hand-held scanner 30 referenced inFIG. 1, CT scanner or MRI. In some instances, the scanning devices maybe located at other facilities, in which case the 3D scans are obtainedat another location and the 3D data is supplied to the workstation 10(FIG. 1) over a suitable communications channel (Internet) or via a disksent in the mail.

The workstation includes a memory storing machine readable instructionscomprising an integrated treatment planning and model manipulationsoftware program indicated generally at 300. The treatment planninginstructions 300 will be described in further detail below. Thetreatment planning software uses additional software modules. A patienthistory module 302 contains user interface screens and appropriateprompts to obtain and record a complete patient medical and dentalhistory, along with pertinent demographic data for the patient.

A module 304 contains instructions for designing custom dental andorthodontic appliances. These appliances could include both fixedappliances, e.g., brackets, bands, archwires, crowns and bridges,surgical splints, surgical archwires, surgical fixation plates,laminates, implants, as well as removable appliances including aligningshells, retainers and partial or full dentures. In one possibleembodiment, the module 304 may be located and executed at the site of avendor of custom orthodontic applicants. The vendor would receive anorder for a custom appliance specifically to fit an individual patient.Module 34 would process this order and containing instruction fordesigning the appliance to fit the individual morphology and conditionof the patient. The vendor would take the appliance design, manufacturethe appliance in accordance with the design, and then ship the customappliance to the practitioner. Examples of how the appliance designmodule 304 might be implemented include the appliance design softwaredeveloped by OraMetrix and described in the published PCT patentapplication cited previously, the customized bracket, jig and wireappliance design software of Ormco described in the issued Andreikopatents (see, e.g., U.S. Pat. No. 5,431,562) and in the published patentapplication of Chapoulaud, US patent publication no. 2002/002841, thetechniques of Chisti et al., U.S. Pat. Nos. 6,227,850 and 6,217,325, allincorporated by reference herein.

The treatment planning software 300 also obtains information on standard(“off the shelf”) dental or appliances from a module 306, which storesmanufacturer catalogs of such appliances, including 3D virtual models ofthe individual appliances.

The treatment planning software includes a module 308 that allows theuser to input selections as to variable parameters that affect thevisual appearance of the patient, as input to a craniofacial analysismodule 328 described below. The variable parameters include patientfactors: age, weight, sex, facial attributes (smile, frown, etc.). Thevariable parameters also include parameters affecting the teeth,including texture (color), position, spacing, occlusion, etc. Thevariable parameters further include various illumination parameters,including reflectivity of the skin, ambient light intensity, and lightdirection. These parameters are accessed though appropriate icons on thescreen display, such as the icons shown in FIGS. 4-7, and pop-updisplays that appear that prompt the user to enter or vary the selectedvariable parameter.

The treatment planning software further uses a diagnosis and simulationmodule 310 that displays diagnosis data graphically and/or in reportformat. This diagnosis data includes teeth position, 3D face and smileappearance, and various facial attributes.

The software further includes third party practice management software312. Information about treatment planes generated by the craniofacialanalysis module 328 is input to the practice management software 312.Based on the treatment plan, this software generates the most productivescheduling of appointments for the patient. The practice managementsoftware 312 also generates reports, provides insurance and benefittracking, and supports electronic claims filing with the patient'sinsurance company. Preferably, the practice management software providesa flexible scheduling of patient appointments based on progress oftreatment of the patient's craniofacial anatomical structures. Theprogress of treatment is obtained from periodically obtaining updatedthree-dimensional information regarding the progress of treatment of thecraniofacial features of the patient. For example, the patient isperiodically rescanned during the course of treatment. A new virtualpatient model is created. Depending on the progress of treatment (e.g.,movement of the teeth to target positions) the patient may be scheduledfor more or less frequent visits depending on their progress.

Referring again generally to the treatment planning software 300, thesoftware includes a 3D model generation module 314 that uses as inputthe 2D and 3D scanning devices. A 3D virtual model of the patient iscreated by module 314, for example, in the manner described previouslyin FIGS. 2 and 3.

The system further includes a custom appliance management module 315.This module provides appliance specifications and 3D geometry data tothe vendor site for the purpose of providing necessary input for thedesign and manufacture of custom appliances, such as custom orthodonticappliances, for the patient. This module also provides updates to anappliance data module 324 for storing custom appliance data within thedatabase. The module 324 is responsible for managing the database of allthe appliances, including custom appliances.

The 3D virtual patient model is supplied to a knowledge database 316.The knowledge database includes a 3D Geometry data file 316 that storesthe 3D geometry data of the virtual patient model. This data is suppliedto a tagging module 322 and a morphable model module 320. The morphablemodel module 320 includes instructions for creating a morphable modelfrom various 3D model samples, using the techniques for example setforth in the article of Blantz et al., A Morphable Model for TheSynthesis of 3D Faces, Computer Graphics Proceedings SIGGRAPH '99(August, 1999).

The tagging module 322 includes instructions for tagging or placingpieces of information regarding the virtual patient model into eachpatient record, which is used for statistical procedures. In particular,the tagged information is supplied to a meta-analysis module 326. Themeta-analysis module implements a set of statistical procedures designedto accumulate experimental and correlational results across independentstudies that address a related set of research questions. Meta-analysisuses the summary statistics from individual studies as the data points.A key assumption of this analysis is that each study provides adifferent estimate of the underlying relationship. By accumulatingresults across studies, one can gain a more accurate representation ofthe relation than is provided by the individual study estimators. In oneexample, the software will use previous patient cases/studies to help inthe craniofacial analysis module 328. For example, surgery cases for“lip and chin” will be one set of independent studies, whereas jawsurgery to correctly position the upper and lower jaw will be another.An orthodontist trying to align the upper and lower jaw will do ameta-analysis with the module 326 in order to see how this treatmentwill affect the patient's lip and chin.

The output of the morphable model from module 320 and the meta-analysisfrom module 326 is provided to a craniofacial analysis module 328. Thismodule takes as input, patient information and the patient 3D virtualmodel to generate diagnosis and simulation data. Based on one or moresimulation results, this module 328, and/or module 330 generates atreatment plan and appliance selection. User involvement is contemplatedin modules 328 and 330. In particular, the user may interact with thepatient information and the morphable model, and vary the parameters308, to simulate different possible treatments and outcomes to arrive ata final or target treatment objective for the patient. The craniofacialanalysis module 328 may include some or all of the treatment planningfeatures described at length in the published PCT application ofOraMetrix, Inc. cited previously.

The software instructions included in the craniofacial analysis module326 preferably includes a set of instructions providing the user withuser interface tools (e.g., icons), for visually studying on the userinterface 16 the interaction of the craniofacial anatomical structuresand their relationship to the external, visual appearance of thepatient. For example, tools may provide a chewing simulation.Alternatively, the tools may provide a smile function in which the faceis morphed to smile, showing the position of the teeth, gums, lips andother structures. These tools simulate changes in the anatomicalposition or shape of craniofacial anatomical structures (teeth, lips,skin, etc.) and show the effect of such changes on the visual appearanceof the patient. As another example, the tools may include tools formodifying the shape or position of one or more bones of the upper andlower jaws, and show how those modifications affect the patient'sappearance and smile.

With reference to FIG. 7, the user would activate one of the icons 35 atthe top of the screen display. The icon may be associated with afunction that would allow the user to reposition the location of theupper and lower teeth. After the user changes the position of the teeth,the user would activate another icon, “smile”, and the face would morphto a smile with the teeth in the new position.

After the patient simulations have been completed and the patient andphysician are satisfied, the resulting data set of teeth position, jawposition, etc. are stored by the diagnosis and simulation module 310 ofFIG. 19. This module 310 preferably includes a routine for storing athree-dimensional representation of said patient's craniofacialstructures (e.g., teeth) in a format suitable for use by a manufacturerof orthodontic appliances. Each manufacturer may have a unique formatneeded for use by the manufacturer, and the routine takes that intoconsideration in storing the data. For example, a manufacturer mayrequire 3D digital models of the teeth in initial and final positions inthe form of triangle surfaces, along with archwire and bracketprescription data.

It is contemplated that the creation and usage of the virtual model mayoccur at the patient care site. In particular, the treating physician ororthodontist will access the scan and photographic data, create thevirtual model therefrom, and perform the treatment planning andsimulation described herein in their own office. Once the treatment planis arrived at, the treating physician can export the virtual patientmodel or some subset of data to appliance manufacturers or specialists,as indicated in FIG. 1.

Alternatively, the virtual patient model may be created at a remotelocation. In this latter example, a third party, such as an appliancemanufacturer, may be the entity that creates the virtual patient modeland makes it available to the treating physician. In this example, thetreating physician will have access to the scanners, X-Ray, digitalcamera, or other imaging device, obtain the required data from thepatient, and forward such data to the third party. The third partyexecutes the instructions to create, visualize and manipulate thevirtual patient model. This model can be transmitted to the treatingphysician for their review and usage. Then, either the third party couldcreate a proposed treatment for review and approval by the treatingphysician, or the treating physician could create the treatment plan.The plan is then transmitted to one or more appliance manufacturers forfabrication of therapeutic devices (e.g., brackets and wires, aligningshells, maxillary expansion devices, etc.)

A treatment plan created from the virtual patient model described hereinmay be one in which only one type of appliances, e.g. fixed ofremovable, is used during the entire course of the treatment. Forexample, the treatment plan may be one in which brackets and wires arethe type of appliance that is used. Or, alternatively, the treatmentplan may be one in which removable aligning shells are the type ofappliance that is used.

On the other hand, the treatment plan might be such that it is a hybridplan requiring the use of different types of appliances during thecourse of the treatment. In the hybrid orthodontic treatment plan, avariety of scenarios are possible. In one type of hybrid treatment plan,different types of appliances might be used at different times duringthe course of the treatment. For example, patient may start out withbrackets and wires and shift at some point during treatment to anapproach based on removable aligning shells. In another type of hybridtreatment plan, different types of appliances might be usedsimultaneously, for example in different portions of the mouth, forexample brackets and wires could be used for certain teeth andtransparent aligning shells uses for a different set of teeth. A hybridtreatment plan may be chosen right from the beginning, or it may beintroduced dynamically at any stage during the treatment course.

To develop a hybrid treatment plan, the treatment planning software willpreferably include features of the appliance design and treatmentplanning software of the manufacturers of the appliances that are usedin the hybrid treatment. As one example, the treatment planning softwaremay include the wire and bracket features of the OraMetrix treatmentplanning software described in the published application WO 01/80761, aswell as the treatment planning software described in the AlignTechnologies patents to Chisti et al., U.S. Pat. Nos. 5,975,893 and6,227,850. The software would thus allow the user to simulate treatmentwith brackets and wires for part of the tooth movement to reach aparticular milestone, and also design the configuration of intermediatetooth positions and configuration of removable aligning shells for theremainder of tooth movement. Alternatively, the shape of the aligningshells could be determined automatically via the treatment planningsoftware from the tooth configuration at which the shells are firstintroduced to the patient and the final tooth position in accordancewith the teachings of the Chisti et al. patents.

FIG. 20 is an illustration of the integration of the patient dataacquisition, treatment planning and appliance design functions that arefacilitated by a preferred embodiment of the unified workstation 14. Theworkstation is provided with a plurality of image data sets 400, whichcan include 2D data (e.g., photographs) 402, 3D image data 404 fromvarious 3D image sources, static models 406 of all or part of thepatient's craniofacial anatomy, dynamic models 408 of all or part of thepatient's craniofacial anatomy, color models 410, and possibly othertypes of image data. The workstation 14 includes software 314 (such asdescribed above in conjunction with FIG. 19) that takes any possiblecombination of this image data to produce a virtual patient model 34.From this virtual patient model, the workstation in one possibleembodiment includes one or more treatment planning tools or software 300for planning treatment for the patient. These treatment planning toolscould include specific software provided by vendors of treatmentplanning software or appliances, such as manufacturer #1 software 412,manufacturer #2 software 414, software for manufacturers #3, 4, 5, . . .at 416, 418, 420, as shown. Such software would be operable on thevirtual patient model 34 and associated data sets representing the teethas described at length herein. To provide interoperability of thesoftware on the virtual patient model, the virtual patient model mayhave to have representations of the data that is compatible with thesoftware of various vendors, which is within the ability of personsskilled in this art. Moreover, once appliance designs have been createdby the various species of treatment planning software, the preferredembodiment of the workstation allows export of appliance design, toothposition data or other required outputs to any appliance manufacturer soas to allow the manufacture of a customized orthodontic appliance. Inother words, if the workstation is equipped with OraMetrix treatmentplanning software, such software could output tooth position data,appliance design data and any other required data into a formatcompatible with the manufacturing requirements of any appliancemanufacture. This interoperability of data formats for appliance designis shown by arrows 421. Thus, the workstation provides a conversion orformatting of appliance design data into a data set or output formatspecified by any one of a variety of particular appliance manufacturers.In the illustrated embodiment, the available therapeutics data sets areshown as manufacturer no. 1 data set 422 (brackets and customizedwires), manufacturer no. 2 data set 426 (brackets and wires),manufacturer no. 3 data set 426 (removable aligning shells),manufacturer no. 4 data set 428 (brackets and wires), or still othersets 430. The appliance design data set is then furnished over theInternet to the vendor of such appliances for manufacture of a customappliance. Hybrid treatment plans, as described above, are onepossibility of a treatment plan that may be developed using theworkstation and virtual patient model described herein.

In one possible variant of the invention, the treatment planningsoftware tools 300 are also provided at a remote location and some ofthe tasks of appliance design may be performed as a service by aseparate workstation, such as a workstation of an appliancemanufacturer. In this situation, the virtual patient model 34 could beprovided to the appliance manufacturer, a proposed treatment plan isprepared and furnished to the practitioner, and after the plan isapproved, the appliance manufacturer coordinates the furnishing ofappliance design data to any designated appliance manufacturers that areused to furnish the custom appliance.

In one possible embodiment, the treatment planning software includes aset of instructions that perform a measurement function to measuredistances in two or three dimensions in the virtual patient model, e.g.,arch form shape measurements, and compare the measurements withreference dimensions for an “average” patient of similar age, sex, andrace. These measurements could be obtained in any convenient manner, forexample from textbooks, organizations, practitioners, etc. Thesemeasurement tools would be invoked during the course of treatment tocompare tooth movement and current tooth position with expectedpositions and if deviations occur, the variances could be used asinformation to modify one or more aspects of the treatment plan, such aschange the appliance design.

A specific example of interactive treatment planning with a unifiedworkstation will now be now described in conjunction with FIGS. 21-60.In order to provide the functions and features described in thefeatures, the workstation includes a computing platform (general purposecomputer) having a graphical user interface, a processor and a computerstorage medium containing digitized records pertaining to a patient. Thedigitized records include image data, such as photographs, x-rays, andscan data of the teeth. The workstation further includes a set ofsoftware instructions providing graphical user interface tools forproviding access to the digitized records, such as display andmanipulation of the images or scan data in the form of 3D models. Theworkstation further includes software instructions for execution by theprocessor for facilitating treatment planning for a patient.

While there are various ways in which practitioner may go about theprocess of designing a treatment plan for a patient, in a preferredembodiment of the invention certain specified tools are provided whichallow a treatment plan to be developed in which constraints can beidentified and the treatment plan can be developed without violation ofsuch constraints.

Referring now to FIG. 21, a screen shot from the graphical userinterface of the workstation of FIG. 1 is shown. The workstationincludes a computer memory that stores, and makes available to thepractitioner, records in the form of digital data pertaining to some orall of the following: the patient's clinical history, medical history,dental history, and orthodontic history as well as 2D photographs, 2Dradio graphic images, CT scans, 2D and 3D scanned images, ultrasonicscanned images, and in general, non-invasive and sometimes invasiveimages, plus video, audio, and a variety of communication records, suchnotes, records of office visits, patient letters or communications, etc.All records and images are digitized. The records and images are madeavailable through suitable user interface icons which cause display ofthe images and records on the user interface. The images can be combinedor superimposed to create a virtual patient model that includes surfacefeatures (soft tissue) of the patient in one possible embodiment.

The workstation also further maintains a comprehensive set of computerinstructions providing tools in the form of icons, screen displays,windows, functions and features, accessible through the user interfaceof the workstation to assist the practitioner in planning the treatment.Various types of tools are contemplated; numerous examples are set forthherein.

In FIG. 21, a set of tabs 450, 452, 454, 456 and 458 are provided. Thetab 450 is a patient information tab which provides suitable screendisplays for entering a variety of patient information, such as theirname and address, dental and clinical history, insurance information,diagnostic information, names of other treating or consultingpractitioners, etc. Tab 450 will be discussed in further detail below inconjunction with FIGS. 21A and 21B and FIGS. 54-58. The tab 452 is a tabwhereby the user accesses scan or other image data and accessesinstructions and menus for scanning the patient with an in-vivointra-oral scanner such as described in the previously cited OraMetrixPCT application. Tab 454 is a tab by which the user accesses the digital3D impression of the teeth, obtained from the scanning of the patient.Tab 456 is a case management tab and includes a number of specificavailable screen displays and menus which are shown in the menu 460 inthe lower left of the Figure. The case management tab, and its variousfeatures, is described at length in the following discussion.Additionally, there is a digital treatment planning tab 458 whichprovides further menus, tools and displays by which the practitioner mayfurther move teeth and design the shape and configuration of acustomized orthodontic appliance. An example of the types of menus andtools that are available in the tab 458 is the OraMetrix treatmentplanning software described in application Ser. No. 09/834,412, filedApr. 13, 2001. However, it is possible to provide, in the workstation, asuite of treatment planning software from different appliancemanufacturers in which case the user could access the treatment planningsoftware for whatever appliance manufacturer the practitioner wished touse for treatment of the patient. In this situation, it may be necessaryto format the tooth data in a format compatible with the appliancedesign and treatment planning software so as to ensure compatibilitybetween the various systems that may be installed on the workstation.

In FIG. 21, the user has selected a “treatment strategy” icon 461, whichcauses the display 462 to appear. In this display, there is a field 464for the user to enter high level diagnosis and problem classificationinformation, for example in the form of text. A field 466 is providedwhich provides a matrix format by which the conditions relevant to thepatient's soft tissue, skeletal, and dental anatomy are entered, eachwith respect to vertical, sagittal, and transverse positions, again intext form. The display also includes a treatment strategy field 468where the user will indicate the general, high level approach totreatment, such as any proposed extractions, appliance type, stages oftreatment, etc. These fields 464, 466 and 468, along with displayedimage data for the patient, assist the practitioner in identifying theconstraints pertinent to the treatment planning.

FIG. 21A shows the patient information tab 450, with the slide bar 850moved next to “history”. The screen display shown in FIG. 21A appears,with a field 451 for which the user can enter alerting medicalconditions, such as AIDS or HIV infection, epilepsy, allergy conditions,tuberculosis, etc., along with any explanatory data or comment. In field453, the user is provided with tools to enter general medical conditionregarding the patient by clicking on the drop-down menu as shown, andentering any appropriate data or commentary, as shown.

In FIG. 21B, the user has moved the slide bar 850 to the “Examination”icon, which causes the display shown in FIG. 21B to appear. This screenallows the user to enter dental examination data in field 455, a toothchart 457 where the user clicks on a particular tooth and enters toothdata in fields 459, 461 and 463 as indicated.

After the user has entered the information into the fields 464, 466, 488shown in FIG. 21, the user clicks on one of the other icons in the field460 to continue the process of case management and initial treatmentplanning. At this point the information entered into the fields of FIG.21 is stored in the computer memory of the workstation.

In FIG. 22, the user has now selected a “Midline and Aesthetic OcclusalPlane” icon 470, which causes the screen display 472 to appear. The useruses this screen to evaluate and define both vertical and horizontallines of references such as soft tissue midline, interpupilliary line,etc and also define the dental midlines for the upper and lowerdentition and the aesthetic occlusal planes for both the upper and lowerarches and cant of the occlusal planes These midlines and occlusalplanes are designed relative to the face, which here is a globalreference. These lines are useful references for where you want thepatient's teeth to move.

When screen display 472 is activated, the workstation displays a pair oftwo-dimensional color photographs of the patient, shown as a photo 474with the patient's mouth closed, and a photo 476 with the patientsmiling. The display includes a field 478 where the patient can maintainthe midline that the user marks on the images, as described below, oractivate one of the other tabs indicating treat to upper midline, treatto lower midline, or provide a custom midline. The midline is enteredusing the tools 486 on the right hand side of the screen A region 480 isprovided for the Aesthetic Occlusal Plane (occlusal plane for the frontteeth), which the user can indicate or mark on the images of the patientusing the tools 486 on the right hand side of the screen. The user marksan Aesthetic Occlusal Plane (AOP) for both the maxilla and mandibledentition, and the user is provided with fields 480 and 482 forcustomization of these planes (technically, lines in two dimensions). Atab 484 is provided to create a customized canted AOP with various tabsas shown. Thus, the tools provide the user to mark, among other things,a midline and maxilla and mandible levels and cant of an aestheticocclusal plane.

The display of FIG. 22 includes a set of tools 486 in the form of iconswhich, when selected, allow the user to mark on the images variousvertical and horizontal lines. For example, the user can mark an upperocclusal plane on the photographs of the upper arch of the patient, alower occlusal plane (line) in the lower arch of the patient, andmarking various positions in the upper and lower occlusal planes, e.g.,marking a posterior position of the upper or lower occlusal plane (linein 2D); marking a functional position of the upper or lower occlusalplane; and marking an aesthetic position of the upper or lower occlusalplane.

As shown in FIG. 23, when the user activates the legend “L” icon 488, awindow 490 pops up and a legend appears that explains the acronyms forvarious lines and midlines that the user may mark on the images.

As shown in FIG. 24, the user has activated various icons 486 and hasdrawn on the virtual model of the patient an aesthetic upper occlusalplane (“AU”) 494 and a aesthetic upper perpendicular line (“AUP”) 492 inthe left-hand image, and a treatment upper occlusal plane (“TxU”) 498and a treatment upper perpendicular line (“TxUP”) 496. The lines 492,494, 496 and 498 are all user specified in terms of their location. Thelocation is selected by using the workstation mouse, moving the cursorto the location where the user wishes to draw the midlines and occlusalplanes, and clicking the mouse.

Referring now to FIG. 25, after the user has proceeded to mark thelocation of midlines and occlusal planes, the user clicks on the icon500, at which point the “Occlusal Plane and AP [Anterior, Posterior]Positions” screen display 502 appears. The display 504 shows otheraspects of the patient virtual model, here a two-dimensional lateralx-ray view of the face, including teeth and jaws. Shown on the display504 are two of the occlusal planes (line in 2 dimensions) the user haspreviously entered from the screen display of FIG. 24: a normal occlusalplane (“NO”) 506, and a Treatment Occlusal Plane Upper (TxOU) 508,indicating the occlusal plane to which the teeth are designed to bealigned with as a result of treatment of the patient. These lines can besegmented into three separate lines, one for the posterior, functionaland aesthetic. The screen display includes a set of icons 510 providingtools for the user to mark various occlusal planes and locations thereofin two dimensions, as well as a Legend icon 520. The display alsoincludes a region 512 whereby the user can modify the location of theaesthetic occlusal plane (front teeth, teeth 1-3), a functional occlusalplane (teeth 3-6), and a posterior occlusal plane (rear teeth, teeth6-8), for both the upper and lower jaws. The axis of cant of theocclusal plane can be changed by rotating around a predetermined centeror fulcrum of rotation. Also, the A/P position and inclinations and thevertical relations of the incisors with respect to the occlusal planecan be represented by animating teeth as shown in FIGS. 56, 57. Thedesired position of the incisors can be planned. The position of theincisors also drives the change in the position of the soft tissue, e.g.lips. Any changes can be compared against a normative database ofpositions of various craniofacial structures. A complete 2Dcephalometric analysis is thus possible.

As shown in FIG. 26, when the activates the legend icon 520, the window522 pops up and provides a legend for the acronyms which accompany theocclusal planes that are shown on the x-ray image 504. The variousocclusal planes 522 are accessed by activating the icons 510 at theright hand side of the screen display, and using the mouse to indicatetheir location on the image 504.

Referring to FIG. 27, the user has finished marking the midline, theocclusal plane(s) and making any anterior/posterior adjustments, andproceeded activate the “Reference Tooth” icon 550. This action causesthe display 552 to appear. The display 552 includes a reference toothselection field 556. The display also changes to show a panoramic X-ray554 to appear, showing all of the patient's teeth to appear in a line,along with associated soft tissue and surrounding bone structures. Theuser makes a judgment decision as to which tooth or teeth should bemoved the least (or not at all), and selects this tooth (or teeth) to bethe reference tooth or teeth. This action is completed by the usermoving the mouse cursor to the tooth or teeth in the field 556 andclicking the tooth they wish to select as the reference tooth. Here, theuser has selected tooth 3 on the upper right hand side as the referencetooth, as indicated at 558. Any changes in crown position in twodimensions or root positions are seen and transferred into thethree-dimensional data sets.

FIG. 27A shows a screen display with the display showing a flattened 3Dx-ray of the teeth flattened such that all the teeth lie in a twodimensional line, with each tooth having a tooth axis indicated by aline segment having one end terminating at the cusp or tip of the toothand the other end of the line segment terminating at the end of thetooth root. This screen is displayed simultaneously with the field 556showing the reference tooth selected. The user, having inspected theaxes of the teeth and their relationship to other teeth via the X-ray,may select a different reference tooth by simply clicking on a differenttooth in the field 556. Typically, the user will select a referencetooth in which the axis of the tooth does not move during the course oftreatment, and the displays of FIG. 27 and FIG. 27A facilitates thatselection. The screen display of FIG. 27A facilitates the measurement ofthe tooth axes to make the reference tooth selection.

The treatment planning process continues by using the graphical userinterface to align two dimensional images of the patient, e.g. x-rays,with three-dimensional virtual teeth models. In this manner, the userprogresses from two-dimensional treatment planning to three-dimensionaltreatment planning. One possible embodiment is shown in FIGS. 28-31. InFIG. 28, the user has selected the “Alignment 2D-3D” icon 570, whichcauses the screen display 572 to appear. In this display, a 3D virtualmodel of the teeth 116 appears on the display, superimposed over a twodimensional X-ray photograph 505. The 3D model of teeth 116 is createdby suitable scanning techniques, such as in-vivo scan of the dentitionor a scan of a model of the dentition, as described previously. Afterthe scan is obtained, the teeth are separated from surroundingstructures and represented in the computer as individual, individuallymoveable, virtual tooth objects. The display includes navigation andother icons by which the user can rotate the model 116 in any desiredorientation, show only one or both arches, select or deselect fordisplay gingival tissue, occlusal planes, etc. The 3D tooth models arescaled so that they coincide in size with the size of the teeth in the2D image. The superposition shown in FIG. 28 could be either performedmanually, or possibly automatically with suitable pattern matchingalgorithms to identify tooth objects in the 2D image and align the 3Dtooth objects with the teeth in the 2D image.

In FIG. 28, the functional occlusal plane 508 is displayed together withthe teeth and the x-ray. Whereas in FIG. 24, the upper occlusal planewas shown as merely a line, in FIG. 28 the occlusal plane 508 isrepresented in two dimensions but it actually is also represented inthree dimensions in FIG. 43. Thus, the original 2D representation istransferred to a surface in three dimensions. The user is able to viewthe arrangement of the teeth relative to the bone in any orientation,such as front perspective or side perspective.

The arrangement in FIG. 28 facilitates the user understanding therelationship of the 3D teeth with respect to the soft tissues as well asbone. The 3D plan of the teeth can be oriented relative to the occlusalplane 580 that the user has defined.

In FIG. 28A, the user has activated icons 574 for display of both archesand a midline plane 581.

In FIG. 29, the user has selected for display both the upper and lowerarches by activation of icon 602, and treatment occlusal planes 606 and608 for the upper and lower arches. The occlusal planes 606 and 608 areactivated by selecting the icon 604. Note that in FIG. 29, the occlusalplanes are rendered and displayed in three dimensions as athree-dimensional surface, whereas, initially in FIG. 24, the midlineand occlusal planes were rendered in two dimensions. The threedimensionality of the planes 606 and 608 is hard to see in FIG. 29, butbecomes more apparent when the model of the teeth is rotated or viewedfrom an orientation that is not so closely in line with the planes 606and 608.

In FIG. 30, the user has activated icon 610 which causes a mid-sagittalclipping plane 612 to appear. The location where the clipping plane 612intersects the front teeth in the upper and lower arches is shown by theshaded areas 614. The clipping plane can be moved over the arch to viewthe teeth in any cross-sectional view, using navigation icons. The viewshown in FIG. 30 allows the user to judge subjectively the relationshipbetween the upper and lower incisors, and compare that with the 2Dviews, for example, from a 2D X-ray.

As shown in FIG. 31, the user can adjust the position of the clippingplane and thereby change the location at which the plane intersects theupper and lower arches, as will be appreciated from a comparison of theshaded areas 614 in FIG. 31 with the areas 614 in FIG. 30. The useradjusts the position of the upper and lower incisors in the clippingplane illustration to match the position that was determined in the 2Dlateral view in FIG. 57.

When the user is satisfied with the 2D-3D aligning step, the userproceeds to additional tasks, including space evaluation, and spacemanagement tasks by which the user first evaluates how much space isneeded to align the teeth and then how he wishes to manage the space.The user further proceeds with the design of an desired arch form. Thisis done for both arches, typically the mandible first and then themaxilla. However, at any time the user can view both arches together byactivating a hide/display icon. To proceed to these tasks, in theillustrated embodiment, the user selects a mandible space managementicon 620, as shown in FIG. 32, which causes the screen display 622 toappear. The screen display 622 includes a plurality of icons 624 whichare used for hiding and displaying various aspects of the virtualpatient model, soft tissue, occlusal planes, and other features of thesoftware. The central region of the display 622 is used to display the3D virtual teeth 116. The display 622 also includes a lower region 626,where the user can activate an Arch Form tab 636, as shown, or othertabs, including a Vertical Positions tab 628, a A/P positions tab 630,and an Extractions and Stripping tab 632. The arch form tab 634 includesan area indicating that the user has selected a customized arch form.However, by activating the drop down icon 638, the user can scrollthrough and select pre-defined arch form types that are stored in theworkstation, and adapt tooth position to standard arch forms. At alltimes, the user is able to interactively move any tooth or teeth on thegraphical user interface relative to the desired arch form, by clickingthe tooth to select the tooth and then dragging the tooth with the mouseto a new position.

Space analysis can be dynamically evaluated by affecting the followingparameters: midline, arch form, A/P position, tooth position, thereference tooth, tooth size, spatial distribution of the teeth in thearch and by appliance prescription, either selectively or in tandem.Furthermore, space management can be effectuated by simulation ofinterproximal reduction, buildup of the tooth, extraction, distal andmesial tooth movement, expansion of the jaw, axial inclination anglechange, rotation change, overject and overbite change, appliance choice,adjustment of inter-arch relationship, or selectively maintainingcrowding.

The tab 634 further includes measurement tools 640 which provide cuspiddistance measurements and inter-molar distance measurements for thecurrent tooth positions displayed on the screen. The user can also setpoints anywhere on the virtual model and activate an icon to get adistance measurement, or invoke a graph tool as described elsewhere.FIG. 32 also shows a display 644 that provides an arch lengthdiscrepancy measurement (in terms of mm) which indicates, given thecurrent virtual tooth positions, whether there is sufficient length inthe arch (positive values) or whether some interproximal reduction,tooth rotation, extraction, distal movement of the molars, uprighting ofthe molars, changing of the torque of the teeth, changing the A/Pposition of the incisors, expanding the arch form, maintaining selectivecrowding, adjusting the level of the occlusal plane or the midline,axial inclination of teeth, overjet or overbite or other action isrequired to fit the teeth in the arch (negative values). The left-rightbalance in the arch length discrepancy can be changed by interactivelyselecting the midline and moving the midline either to the right orleft.

In FIG. 32, the presenting malocclusion is seen. The user has indicatedthat they wish to hold the malocclusion as indicated at 646. In FIG. 33,the keep lateral segments tab 650 has been activated. This featureimmobilizes the lateral segments (premolars and molars) and allows forthe alignment to occur only through the change in the arch formanteriorly. The user can define the incisor position based on the lowerincisor based on where it was initially. The user can hold the buccalsegment (basically holding the lateral segments) then impose no movementand you allow for full alignment of the anterior teeth. What this hasdone is tell the practitioner how much space is needed in the archlength and at the same time the display provides the inter-molar andinter-cuspid width based upon holding the posterior teeth fixed. Inother words, as the user changes the constraints in terms of toothmobility, the user is provided instantaneous measurements in terms ofarch length and the new positions of the teeth that are allowed to move.The user can selectively immobilize teeth or allow their free movementin three planes of space, either individually or in groups.

The teeth are moved to a more ideal position. This action changed thecuspid distance from FIG. 32, but did not change the molar distance. Thelength discrepancy tool 644 indicates that the right and left archlength has increased from the previous values, which would require someinterproximal reduction, extraction, or some other action to fit theteeth to the arch length.

In FIG. 34, the user has selected “full aligned”, as indicated at 660.Both the cuspid distance and the molar distance have changed, asindicated by the measurement tools 640. The length discrepancy, asindicated by the display 644, has now returned to zero, indicating thatthe arch form is “good”. “Fully aligned” allows free movement of all theteeth along the pre-selected arch form. If some arch length discrepancyremained, the user could simulate a change in arch form, midline,occlusal plane, rotation of teeth about their axis, extraction etc. torectify the situation. If the user wants to customize the shape of thearch form they activate the slide line tab, discussed later.

In FIG. 35, the user has activated icons 670 and 672, which causes thedisplay to show both the original position of the teeth (areas with darkshading indicated at 674) and the new position as a result of themandible space management exercise (white areas, indicated at 676. Thiscolor coding helps the user visualize the tooth movement that will occurin accordance with the proposed treatment plan. This feature ofreference back to the original malocclusion is available at any time andin any plane, both in 3D or 2D images or combination.

In FIG. 36, the user has activated icon 680 which causes a slide line682 to appear. The slide line can positioned at different levels, suchas the bracket level, the level of the cusp tips, or the level of theinterproximal contact points.

Arch length discrepancy can be defined at various levels, includingcontact points, cusp tips and at the level of brackets, based upon thelocation of the slide line that is chosen. Then, the effect of bracketprescription on the dentition is also modeled in defining the positionof the teeth in the arch, thus providing the clinician with a method ofunderstanding the effects of his appliances on arch length inadequacy.

The slide line 682 is a tool that assists the user in changing the shapeof the arch form. The slide line 680 includes anchor points 683 spacedalong the length of the slide line 682, which are affixed to labialsurfaces of the teeth in the positions shown, The slide line 682 alsoincludes points 681 equidistantly spaced from the anchor points, whichthe user manipulates to cause the slide line to bow out or in relativeto the teeth, and thereby change the shape of the arch form. For examplethe user would click on one of the points 681 and drag the point 681 outaway from the slide line, which would cause the slide line to bowoutwardly towards the point 681. The clamping or anchor points can bemoved by the user anywhere along the slide line. The slide line (as wasthe case with the midline) allows for designing asymmetric arch forms.Whenever the user wishes to compare the proposed arch form with theoriginal tooth position, they activate an icon at the top of the screenand the original tooth position is also shown, with the difference inposition shown in a contrasting color.

FIG. 36A shows a screen display when the treatment planning icon 900 isactuated. This icon takes the user to addition treatment planningsoftware which enables a user to further define the shape of the archform, move teeth relative to each other, and design customizedappliances. Many of the features shown in the display of FIG. 36 A aredescribed in the published PCT application WO 01/8076, and thereforewill not be repeated here. The user has highlighted the “TARGETPARAMETER” icon 902, which allows the user to customize theconfiguration of the desired target arch form. Here, in area 906, theuser can scroll through a series of available general arch form types.In field 903, the user is provided with various tools to make the archform symmetric (or not symmetric), enter in values for transverse,sagittal or vertical movement, or change the relation of the overbite oroverjet. As shown in FIG. 36A, the slide line feature 682 is alsodisplayed, which the user can use as described previously to change theshape of the arch using the slide line features.

In FIG. 37, the user has activated an icon to show a midline plane 690to appear. By interactively moving the midline (using click and dragtechnique), the user can change the orientation of the midline relativeto the teeth in the arch and thereby change the right/left values shownin the length discrepancy icon 644. The midline is shifted slightly tothe patient's right. The lateral segments are held fixed while themidline shift is only carried out by the anterior teeth. If “fullaligned” had been checked the all the teeth would move along the slideline to accommodate the new position of the midline. Note also in thisFigure that the new position of the teeth can be measured against theoriginal as indicated by the lightly speckled pattern of the posteriorteeth, which indicates little movement of the lateral teeth from themalocclusion position (the difference in position indicated by the darkor shaded spots on the teeth).

In FIG. 37A, the user has selected a parameter tab 690, which includesicons 691 for midline shift (left or right), the measurement toolsproviding cuspid and intermolar distance, and a tool 694 indicating theamount of over bite and overject, given the current configuration. Thisdata is obtainable since the position of the upper arch relative to thelower arch is known in the in the computer, and the position of theincisors in both arches is known in three-dimensional space. Thus, thedisplay allows the user to interactively move the teeth and immediatelyrealize the effect of tooth movement on overbite and overjet. You canalso adjust overjet and overbite parameters to see their effect on thearch length inadequacy.

By activating icon 692, the user can manage the spacing between teeth byhaving all spacing between teeth to be equal. By activating icon 693,the user invokes a collision detection algorithm that prevents the userfrom moving teeth in a manner such that a tooth collides with anothertooth, either in the same arch or in the opposing arch. The softwareallows for interproximal reduction by morphing the tooth shape to matchthe available space, using a simple morphing algorithm that shrinks thetooth in two or three dimensions.

In FIG. 38 the user has invoked the simulation of the midline plane 690.By movement of the midline plane left or right they can correct thelength discrepancy (left or right) for the arch.

In FIGS. 39 and 39A, the user has activated a gingival display icon 700which causes the display to show both the 3D virtual teeth 116 and a 3Dmodel of the gingival tissue 702. This icon can be activated at any timein the space planning screen. The model 702 of the gingival tissue canbe obtained from a scan of the patient's mouth with an intra-oral 3Dscanner as described previously, or it could be obtained from a scan ofa physical model of the patient. The gingival tissue 702 is separatedfrom the scan data of the teeth 116 so that the two types of objects canbe modeled as separate 3D objects and thus displayed either alone ortogether. In FIG. 38, the proposed treatment indicates substantial bumps703 on the lingual side of the teeth. The gingival tissue is morphed toshow the effect of tooth movement to this position on the gingivaltissue. Since gingival tissue closely follows the underlying bone, thebumps 703 may indicate that the proposed tooth movement is incompatiblewith the patients mandible bone structure. This could be confirmed forexample by zooming in on the location of the bumps 703, invoking thedisplay of underlying bone structure from an X-ray or CT scan stored inthe workstation, and examining the superposition of the virtual toothover the underlying bone. In FIG. 39A, the user has moved tooth 701slightly and the gingival tissue 702 “morphs” to follow the position ofthe tooth.

In FIG. 40, the user has activated navigation icons on the display torotate the model of teeth+gingival tissue to a new orientation. The userhas also activated the A/P positions tab 630, which allows the user toconstrain either a singular movement of teeth or a group movement ofteeth in the A/P direction. The display shows a field 710 which allowsthe user to either maintain or interactively change the A/P position ofthe right molar, a field 712 that allows the user to interactivelychange the A/P position of the central incisors, and a field 714 thatallows the user to interactively change or maintain the A/P position ofthe left molar. As shown in FIG. 40, the user has interactively changedthe anterior position of the central teeth by an amount of 1.5 mm, andthe change is simulated by moving the central teeth anterior wise by 1.5mm and keeping the back teeth fixed and showing the new position on thescreen display. The 1.5 mm movement may not be sufficient given the 0.9mm arch length inadequacy remaining.

In FIG. 41, the user has selected an icon 720 that causes the display toshow an occlusal plane 722, along with the position of the teeth 116 ina proposed arrangement. The areas where the occlusal plane intersectsthe teeth 116 is shown as white areas 724 of the teeth.

As shown in FIG. 42, the user has re-oriented the teeth to a side viewand activated the icons to show both the teeth 116 and the treatmentocclusal plane 722. This view gives the user a better view of the pointsof intersection of the occlusal plane with the teeth.

In FIG. 43, the user has activated the extractions/stripping tab 632.This causes the display to show the teeth of the lower arch, and a rowof numbers corresponding to each tooth in the lower arch. The displayalso includes a check box 730 below each number. If the user un-checksthe box 730, the corresponding tooth disappears to simulate anextraction. The numbers above the check boxes indicate the spacingneeded in each half of the arch to meet the arch length requirements andreduce the arch length discrepancy to zero.

The display also includes an icon 721, which, when activated, all theteeth in the arch are moved in three dimensions such that they justtouch the occlusal plane. This is an automatic function, since thelocation of the teeth and teeth cusps are known in three dimensions, andthe treatment occlusal plane is also known in three dimensions.

FIGS. 44 and 44A show the user activating a grid icon 740, which causesthe display to show a grid 742. The grid, which is in units ofmillimeters, gives the user an additional tool to gauge distancesbetween teeth (e.g., inter-molar width, inter-canine width, etc.). InFIG. 44, the numbers in the field 632 above the check boxes areessentially evenly distributed among the teeth in the left hand side ofthe arch, and in the right hand side of the arch, indicating that theinterproximal reduction is symmetrically and evenly distributed betweenthe teeth. The teeth can be morphed accordingly to simulate the changein shape either through reduction in size or buildup in size asnecessitated by the treatment. By un-checking any of the boxes in field632, the user can simulate a tooth extraction and move teeth to managethe gap in space on a tooth by tooth basis, or have the spacedistributed evenly between the teeth, etc. Once the user is satisfied,they can go to the screen of FIG. 28 and see where the teeth arerelevant to the bone.

FIG. 45 shows the virtual model of the teeth of the lower jaw. Becausethe teeth are shown in a plan view it may be difficult for the user tosee the texture and surface characteristics of the teeth. Thus, the userinterface preferably includes an icon 744, which when activated allowsthe user to change the simulated illumination of the teeth, i.e., becomebrighter, or come from a different place, in order to more clearly showthe surface characteristics of the teeth.

FIG. 46 shows the user activating the vertical position tab 628. Thistab includes a field 750 in which the user can either maintain themalocclusion vertical position of the right molars, or change theposition by any user specified amount either towards or away from theocclusal plane. Similarly, the tab includes a field 752 where the usercan either maintain or change the vertical position of the front teeth.A field 754 allows the user to maintain or change the vertical positionof the left molars.

FIG. 47A shows a contact points feature that provides the user of agraphical display of the contact points between teeth. The user hasactivated the arch form tab 634, but the contact point function can beinvoked at any time while the user is in the “Mandible Space Management”routine. The user has activated icon 760, which signifies that the userwishes to inspect and measure and the contact points between the teeth.When icon 760 is clicked, points 762 appear on the teeth which indicatethe points of contact between the teeth. The points 762 are joined bystraight lines 764, which indicate the shortest distance between thepoints. By placing the cursor on a particular tooth, the display showsthe tooth number and distance along the line in millimeters. This givesus the true measure of the widest points in the teeth. These contactpoints are automatically identified and the measurements are alsoautomatic. The user can also change the location of the points to makeadditional measurements of the teeth. For example, for tooth 41, thedistance is 5.5 mm.

FIG. 47B shows the teeth of FIG. 47A rotated and displayed in a neworientation. By using the camera navigation icons 766, the user can zoomin or rotate the teeth to have a new viewpoint as desired. This enablesthe user to see more readily, in three dimensions, how the teeth areoriented.

FIG. 47C shows the user activating an icon 770 which uses a featureextraction algorithm to highlight for the user the marginal ridges andindicate the distance along the marginal ridges. The marginal ridges andassociated lines connecting them across the teeth are shown in FIG. 47 Cas lines 772. Other feature extraction tools are possible, includingicons for showing cusp tips and cusp fossa.

Usage of the contact points feature of FIGS. 47A and 47 B is shown inFIGS. 47 C and 47C. In FIG. 47D, the user has displayed the malocclusionteeth model 116 along with the lines 772 indicating the marginal ridges,together with the occlusal plane 780. Note that the lines 772 have apronounced step relationship. In FIG. 47E, the user has interactivelymoved the teeth in the model 116 so that the teeth just touch theocclusal plane 780. Note that the step relationship in the lines 772 hasessentially disappeared. The user can either move a tooth individuallyor select a group of teeth and move them as a group. Once the molarshave been moved as shown in FIG. 47D the user will typically proceed tosetting up the incisor position. To do this, the user may wish to invokethe clipping plane feature discussed elsewhere in this document to cutthrough the teeth and define the upper and lower incisor relationship intwo dimensions, and then in three dimensions.

After the user has completed space management for the mandible using thetools in the previous figures, the user proceeds to maxilla spacemanagement using the tab 790. Similar screen displays as shown in theprevious “Mandible Space Management” figures are provided to the userand thy perform space management tasks for the upper arch.

After completion of maxilla space management, the user proceeds to clickon a “Space Management (Mandible and Maxilla)” icon 792, and the screendisplay of FIG. 48 appears. In this display, the user is provided withthe same set of icons across the top of the display, allowing the userto hide or show the upper or lower arches, gingival structure, occlusalplanes, contact points, X-ray or other image data, etc. Here, the userhas selected for display both arches in the occlused condition and thetooth model 116 is displayed. This display includes a field 800 wherebythe user can shift the midline of both the upper and lower arches rightor left by pressing the R and L buttons with the mouse. The midline thenshifts, with the amount of the shift indicated in numerical value. Thedisplay includes an overbite icon 804, which indicates the amount ofover bite in mm. The user can user the icon to change interactively theamount of the overbite. Similarly, using icon 802 the user can changethe amount of the overjet. The tools 806 and 808 provide the lengthdiscrepancy for both arches. FIGS. 48A and 48B shows similarfunctionality being available during activation of the “DigitalTreatment Planning Icon 900 and its associated displays; in thesefigures the Target Parameter tab 902 is selected and the user ischanging values for overbite and overjet via the numerical value entryfields 910 and the results are immediately visible on the display.

In FIGS. 49 and 49A, the user has activated the icons across the top ofthe display to simultaneously display both a 2D panoramic X-ray 810 ofthe teeth and jaw as well as the 3D model of the teeth, but with theteeth 116 spread out and represented in two dimensions in registry withthe 2D panorama X-ray. The teeth models that are shown in FIG. 49represent the tooth positions in a proposed treatment. In FIG. 49B, theuser has unchecked the X-ray icon and only the teeth are displayed. InFIGS. 49, 49A and 49B, the user activates the icons to superimpose the3D model in registry with the 2D x-ray. The user thus is able to figureout the axis inclinations of the teeth and can revisit the selection ofa reference tooth (see FIG. 27) if appropriate. Any changes in toothposition and its effect on root position can be evaluated in the 2D/3Dmodel of the panorex radiograph, CT scan, etc. plus superimposed crowns,as in FIGS. 49A and 49B. From these views, the user can quickly arriveat a proposed setup by aligning (interactively moving) the 3D teeth upand down to find the ideal position, selecting or designing an arch formand midline, and then wrapping the teeth around the desired arch form inthree dimensions. When viewing the resulting proposed set up, e.g., inFIGS. 48 and 48A, the user is again given arch length inadequacy from athree dimensional model and proceed to modify the set up to eliminatethe arch length inadequacy using the techniques described previously. Asproposed modifications are made, the user is instantly provided with newvalues for arch length inadequacy, and can immediately determine whetherthe proposed modifications are sufficient to remove remaining archlength inadequacy.

In FIG. 50 the teeth 116 that are shown are the malocclusion or originalarrangement of the teeth. Thus, FIGS. 49 and 50 show that the user cantoggle back and forth between initial tooth configuration and proposedtreatments for the patient. Any changes in any one environment changesthe values in the other environments.

In FIG. 51 and FIG. 51A, the user has used the navigation icons torotate the view shown in FIG. 50. The user is provided with excellentvisual aids to view how the teeth line up with the tooth roots.

FIG. 52 illustrates that the user can unclick the icon causing the X-rayto appear on the screen and simply perform space management for both theupper and lower arches using the virtual teeth models. As shown in FIG.52, the teeth 116 are arranged in a line and seen from an anterior view.FIG. 53 shows the view of the teeth 113 from the opposite direction.

After the user has completed the task of managing space between thevirtual teeth in the proposed arrangement; the user is able to cycleback and repeat any of the previous steps by activating the icons on thelower left portion of the display and entering into the appropriatedisplays and making further adjustments in the proposed arrangement.

The user can then access the rest of the treatment planning software,such as the software indicated by tab 458 (FIG. 21) and proceed withadditional treatment planning procedures. Finally, when they arefinished, the user selects a finalized proposed treatment plan fortreating the patient. This updates the patient's prescription and isstored in the workstation. The display of FIG. 48 shows how thepatient's teeth will look at the end of treatment. Adjustments ininter-arch relationships, as shown in FIGS. 28 and 29, either as aresult change in overjet or overbite, or change in relationship of thejaws, are either tooth driven or jaw driven. These actions change archlength inadequacy values. These can all be simulated on the userinterface.

FIG. 54 shows the user entering the patient information tab 450, wherethe patient has access to dental clinical examinations and dentalradiographic examination. The upper left hand portion 848 includes aslide bar 850 that allows the user to access various fields, includingdemographics, patient history, examination (notes), photographs, andX-rays. Additional fields are made available by activating the “Next”icon. The user has moved the slide bar to X-rays. In the display 854,the user is provided with the X-rays that are currently stored for thepatient, which include a later X-ray of the face (“Lateral Ceph). Theportion 856 of the display shows the X-ray and icons for rotation of theX-ray, changing the brightness and contrast, and displaying a mirrorimage of the X-ray. The user can point and click in any region ofinterest to access dental history or access photo image databases,radiographic images, and so forth. The navigation icons allow the userto rotate, pan, zoom all the X-rays to see them appropriately to checkfor pathology, etc. Also, the user can mark up the X-rays for makingmeasurements in two dimensions, measuring angles, and entering thatinformation into the patient database.

In FIG. 55, the user has navigated to a soft tissue analysis screen fromthe patient information window. Here, the user is allowed to enter intothe workstation via field 860 specific, numerical values for initialmeasurements of soft tissue, desired values for these soft tissueparameters. In the field 862, the user is able to measure or calculateangles, e.g., between various teeth and various planes, angles ofcanting of planes that the user has specified, etc. Again, these anglesare calculated by means of display on the workstation of patient'sX-rays or other image data (including possibly scan data).

The workstation includes a database of “normal” or normativemeasurements for patients of different ages, races, and sexes, for bothsoft tissue measurements as well as all of the angles shown in FIG. 55.The comparison thus leads the practitioner to identify deviations fromnormative measurements. The display shows the normal values in the righthand column 864 of the displays 860 and 862. Thus, as the user isdesigning treatment and entering proposed or “desired” values for any ofthese biological parameter, the screen display simultaneously displaysthe “normal” values for a patient having the same (or approximately thesame) characteristics as that of the patient.

Additional feature extraction algorithms that the workstation preferablyprovides besides the marginal ridge and contact points featuresdescribed previously, include algorithms for identifying tooth cusps andfossa of the teeth. Such measurement tools are useful in automaticallyperforming the Bolton tooth discrepancy level and Angel classificationmethods.

One of the unique features of the software is that the measurementfeatures described herein allow the practitioner to determine the Boltontooth size discrepancy.

Bolton Analysis

A method developed by W. Bolton (1958) for the evaluation of mesiodistaltooth size discrepancies between sets of corresponding maxillary andmandibular teeth. The analysis distinguishes between the “overallratio,” which involves all permanent teeth except the second and thirdmolars, and the “anterior ratio,” which encompasses only the sixanterior teeth of each jaw. For this analysis it is assumed that therelatively smaller tooth material is the correct one. A table ofstandard values lists the tooth width value in the opposing arch that isideally related to this given correct value. The difference between theideal and actual dental width in the arch with the excess value gives anestimate in millimeters of the severity of tooth size discrepancybetween the arches.

Tooth Size Discrepancy (Bolton Discrepancy)

Incongruity between the sums of the mesiodistal tooth sizes of sets ofcorresponding maxillary and mandibular teeth, is determined by theBolton analysis. A discrepancy could involve the “overall ratio” (whichencompasses all permanent teeth except the second and third molars) orthe “anterior ratio” (which includes the six anterior teeth of each jaw)and is identified as a maxillary or mandibular excess or deficiency.Only deviations that are larger than two standard deviations areconsidered to be of potential clinical significance.

A tooth size discrepancy may cause difficulties in achieving an idealoverjet and overbite or arriving at a good intercuspation during thefinal stages of orthodontic treatment. Different ways to address such aproblem include extraction of teeth in the arch with the excess toothmaterial (usually one mandibular incisor), interproximal stripping,compromising the angulation of some teeth so they can occupy a larger ora smaller space in the arch, or increasing the mesiodistal tooth size inthe arch with the deficiency in tooth material (build-ups).

The present software provides measuring tools for measuring theseparameters and conducting this analysis h (using the contact pointsalgorithm described and illustrated previously). Moreover, theworkstation includes a database of normative or normal ratios forpatients. The user compares the ratio for the patient, obtained directlyusing the measuring tools, and compares the result with the normativevalues from the database in the workstation. The difference is displayedfor the user. The result is the Bolton tooth size discrepancy and isuseful in treatment planning and allows the user to measure the totalform or shape of the teeth.

Another feature provided herein is the so-called “Angle classification”,which is a measure of how closely the upper and lower arches fit in anocclused condition. The classification system is as follows.

Class I Malocclusion (Neutroclusion)

A malocclusion in which the buccal groove of the mandibular firstpermanent molar occludes with the mesiobuccal cusp of the maxillaryfirst permanent molar. The term “Class I” is sometimes used incorrectlyas a synonym for normal occlusion, although in reality, it onlysignifies a normal relationship of maxillary and mandibular first molarsin the sagittal plane.

Class II Malocclusion (Distoclusion, Postnormal Occlusion)

A malocclusion in which the buccal groove of the mandibular firstpermanent molar occludes posterior (distal) to the mesiobuccal cusp ofthe maxillary first permanent molar. The severity of the deviation fromthe Class I molar relationship usually is indicated in fractions (ormultiples) of the mesiodistal width of a premolar crown (“cusp” or“unit”)

Class II Malocclusion, Division 1

A Class II malocclusion with proclined maxillary incisors, resulting inan increased overjet

Class III Malocclusion (Mesioclusion, Prenormal Occlusion)

A malocclusion in which the buccal groove of the mandibular firstpermanent molar occludes anterior (mesial) to the mesiobuccal cusp ofthe maxillary first permanent molar. The same conventions as describedabove are used to indicate the severity of deviation from a Class Imolar relationship.

Angle Classification

“Subdivisions” (left or right) are used in asymmetric situations toindicate the side that deviates from a Class I molar relationship.

The workstation software features measurement tools to directly makethese measurements (by measuring the distance between cusps and fossa ofopposing teeth). The results can be quantified and displayed to a user,and compared to normative values in a database. Additionally, the valuescan be classified in accordance with the Angle classification system,e.g., Class I, Class II or Class III. The resulting display ofclassification is useful for interdigitation or changing the spacingbetween the opposing teeth.

Another feature of the software is that it allows the teeth in either orboth arches to be displayed as semi-transparent objects, which allowsthe user to view through the teeth to see opposing teeth or adjacentteeth. Several possible method of providing semi-transparent teeth is toshow fewer of the points in a point cloud of teeth or fewer triangles ina mesh or triangle surface representation of the teeth.

FIG. 56 illustrate the patient has navigated to a cephalometric markingscreen in the patient information tab, where the user has chosen fordisplay a lateral ceph X-ray of the head. The user has also retractedtwo dimensional template teeth 870 from a library of template teeth andsuperimposed the template teeth over the X-ray. The user has alsoactivated the icon s 872 and 873 which causes an occlusal plane 874 toappear on the display. By activating the icons 876 in the left hand sideof the display, the use can navigate to calculations screens andassociated tools, which provide the user with the ability to calculatevarious parameters, a soft tissue analysis screen, landmark status,display options and other tools. The user can move the teeth in twodimensions, both the upper and lower teeth. The user marks the occlusalplane, and that the user is able to move the teeth relative to theocclusal plane.

In FIG. 57, the user has navigated to a hard tissue analysis screen,wherein the user is provided with tools to mark various hard tissueanatomical locations on the X-ray image. Here, the user has activatedicons to compare initial and desired tooth positions using templateteeth to simulate a proposed treatment of the patient. The dark teeth880 represent a proposed tooth position whereas the light teeth 882represent initial tooth positions. The user can change the proposedtooth position by moving the dark teeth using the mouse by clicking anddragging.

In FIG. 58, the user has navigated to a screen display showing both thehard and soft tissue, with the display 884 proving the user the tools tomark specific soft tissue locations on the virtual patient model,including Glabella, soft tissue Naison, subnasele, mentolabial sucus,etc. By closing out of window 884, the user accesses the window 886where the user is able to enter hard tissue points in a similar fashionon the display. The user is again able to make measurements between anypoints that are marked on the screen and measure corresponding angles.

The treatment planning described in FIGS. 22-26 and 54-60 is essentiallydone in two dimensions. After these initial steps are taken, thesoftware allows the user to undergo more specific treatment planningoperations in three dimensions using the virtual 3D model of the teeth,as described in conjunction with FIGS. 27-53. All changes to dentalposition or bone changes can be translated into changes in soft tissueappearance using morphing algorithms. The icons also allow forstandardized views: side, planar, etc.

After the user has completed the task of managing space between thevirtual teeth in the proposed arrangement, designing the desired archform, and arriving at a proposed tooth arrangement or treatment plan,the user is able to cycle back and repeat any of the previous steps byactivating the icons on the lower left portion of the display andentering into the appropriate displays and making further adjustments inthe proposed arrangement.

The user can then access the rest of the treatment planning software,such as the software indicated by tab 458 (FIG. 21) and proceed withadditional treatment planning procedures. Finally, when they arefinished, the user selects or saves the treatment plan. The process canbe repeated as often as desired and the screen displays are structuredso that the user can navigate anywhere in the displays at any time, andtherefore repeat, as necessary, the aligning steps, the design of thearch, enter additional patient information, access the appliance designfeatures and change the appliance design, etc. Moreover, as the designof tooth finish position dictates or drives the prescription of theappliance, the present treatment planning techniques lead directly toappliance design parameters (bracket and wire position, or othertreatment design, such as staged shell configuration) for treatment ofthe patient.

It will be appreciated that the comprehensive functionality provided bysoftware described herein is fully applicable to a full range ofcraniofacial disorders, and while the preferred embodiment is in anorthodontic context, the invention is certainly not so limited.

It will further be noted that there may be some interdependenciesbetween the constraints, in other words, if the user changes oneconstraint, e.g., occlusal plane, other constraints may also be affected(in 2 and 3 dimensions). Examples of such constraints include A/Ppositions of the incisors and molars, intermolar width, intercaninewidth, amount of overjet, amount of overbite, sagittal relation of theteeth, and lip protrusion.

Most of the user interface tools described herein can be used toevaluate the quality of a set-up, using a series of logical steps.Basically, the set-up is the proposed tooth position or prescription fortreating a patient. The setup evaluation may be performed by thepractitioner that prepared the set up, or it may be performed by a thirdparty. The purpose of the evaluation is several fold:

-   -   a) Determine how close the set up is to the objectives        identified for treatment of the patient;    -   b) Determine how close the setup is to established clinical        standards of care;    -   c) Provide a means by which the proposed set up can be evaluated        in a multipractitioner environment, for example in a situation        where all practitioners in the group have to agree on the        set-up. Obviously, the proposed set up is transportable over a        communications network in the form of a file or else accessible        on a central server where others can access it.    -   d) The evaluation can also serve as a guide by evaluating the        course of treatment and the eventual outcome of treatment and        providing a means to measure the difference between the actual        outcome and the expected outcome. To realize this aspect, the        practitioner would need to periodically obtain updated scans of        the patient during the course of treatment and compare the        current (or final) tooth position with the expected position and        use measuring tools or other graphical devices (shading on tooth        models) to quantify the amount of variance between the actual        and expected position. Obtaining tooth position data during the        course of treatment can be obtained by using the in-vivo scanner        described in the published PCT application of OraMetrix, cited        previously.    -   e) Provide a check on the choice and design of therapeutic        devices. Unless the setup is evaluated correctly any therapeutic        device design (e.g., bracket placement location or archwire        shape) may be wrong. The evaluation provides for dynamic change        of the setup, and resulting appliance design, before the        initiation of treatment.        The treatment planning system described herein provides an ideal        basis for performing these evaluations. More particularly, it        provides a complete and integrated workstation environment and        all the necessary processes and software tools to perform the        evaluation. Image data and actual measurements quantifying the        relationship between the patient's teeth and associated bone and        facial anatomy is literally at the practitioner's fingertips.        The workstation provides ways to systematically look at the        set-up, including providing standardized views (plan, side,        anterior, posterior, etc.). The workstation provides both        measurement tools and a full suite of visualization tools to        both measure the proposed setup as well as model interactively        proposed changes in the set up. Complete user interaction with        the virtual patient and the proposed set up is possible, both in        a single site (on one workstation) and also in a multiple site        situation where the model is shared with different practitioners        (e.g., surgeons, orthodontists, prosthodontists, etc. all        treating the same patient).

While various possible approaches to set up evaluation may be taken, thefollowing is one presently preferred approach.

In the evaluation process, the evaluator checks for the compliance ofthe proposed set up with practitioner developed boundary conditions: themidline, the occlusal plane(s), the arch form, and any other referencepoints such as the patient's face or smile, soft tissue, skeletaltissue, dental tissue, functional movement of the jaw, or other boundarycondition. These references could be either boundary conditions or usedas a starting point. The tools used here are the radiographicexamination records (X-rays), two-dimensional photographs, and initialmodel of the teeth, and the various icons and user interface featuresthat allow the user to access these data points and interactively viewthem and change their position. A key feature here is the ability tosuperimpose the virtual 3D teeth or 2D or 3D image data showing bone androot structures of the teeth, and provide the user to freely navigatethrough the models and images, view them from any orientation, and zoomin or out, etc.

Moreover, the proposed set-up and its relationship to boundaryconditions can be observed in various combinations of hard and softtissue, such as teeth and gingival, teeth and lip or cheek, teeth andbone, or teeth, bone and soft tissue.

The setup evaluation further includes features to evaluate theinter-arch relationship, as described herein. The Angle classificationof the occlusion can be determined, as described above. The degree ofoverjet and overbite in the proposed arrangement can be both visuallyobserved and quantified numerically. Clipping plane features allow forviewing the cross-section of the teeth. The analysis of the inter-archrelationship can proceed by an evaluation of teeth on a tooth by toothbasis, by comparison of a slide line for the upper arch with the slideline of the lower arch, evaluation of the position and axis ofinclination of a reference tooth, display of gingival tissue, or hidinggingival tissue, and evaluation of the contact points between teeth. inthe same arch, the marginal ridges, cusp tips and fossa.

While the evaluation may proceed in any manner, one possible checklist,and order of evaluation of the proposed set-up, is as follows

1. Were the boundary conditions met? In other words, conduct a checkthat the boundary conditions in the set up have been delivered inaccordance with the proposed prescription. This would involve evaluationof the midline (viewing frontal photographs and panorex images), thetooth models and checking the upper and lower midlines. Next, theocclusal planes are checked. Then the axis of the reference tooth orteeth are checked. Then a check of any fixed teeth is made—confirm A/Pposition of the teeth is fixed, the vertical position is held fixed, andthe torques are held fixed.

2. Is the set up ideal? This includes a check of the aestheticappearance of the teeth in the proposed arrangement, in variousperspectives. The set up evaluation includes evaluation or confirmationof the following a) the interarch relationships: occlusion class forboth right and left sides, overjet, overbite, as well as b)tooth-to-tooth relationships: front intra-arch incisor alignment foruppers and lowers (axial alignment, embrasures, contact points), caninetooth intra-arch tooth alignment, lateral intra-arch buccal segmenttooth alignment, and tooth positions and rotations (in/outrelationships, first order rotation alignment, contact points for bothupper and lower arches. The evaluation proceeds to checking the marginalridges, third order torque for the buccal segments and front segments(including left/right torque symmetry).

The user then performs a final check of the setup by comparing thefrontal photographs to the malocclusion and then the setup. Afterconcluding the evaluation, the user indicates their decision by selectedAccept, Modify or Reject. If the user seeks to modify the set up, theycheck Modify and then go back and use any of the tools described hereinto make appropriate changes, and then go through the evaluationchecklist again. When they are finally finished, they check “ACCEPT” andthe proposed setup is saved as such in memory for the workstation alongwith the date the set-up was accepted.

An example of the user staging treatment in stages is set forth in FIGS.59A-59-I. FIG. 59A shows a virtual model of the teeth in a malocclusionwith virtual brackets and wires in a configuration designed to move theteeth to a desired position. The placement of the brackets and wires isprovided via the digital treatment planning procedures available in thesoftware. The field 1000 indicates actual tooth movement during thestages of treatment; in FIG. 59A, the fields are blank since the teethare in the original position. In FIG. 59B, the user simulated a 10percent stage of treatment, that is, 10% of the movement of the teeth issimulated. Such simulations are possible since the workstation knowsboth the initial position of each tooth and the final position of eachtooth in three dimensions. The values in the field 1000 show that sometooth movement has occurred. During treatment, when the practitionerexpects that roughly 10 percent of tooth movement should have occurred,the user scans the patient, stored the virtual model of the teeth in theworkstation, and compares actual tooth position with the planned toothposition by hiding or showing the relevant tooth models and orsuperimposing them and using color coding or shading to indicatedeviation between actual and expected positions.

FIG. 59C shows the model 116 of the teeth in the 10 percent stage, withthe brackets and wires hidden.

FIG. 59D shows the shows the expected tooth position in the mandibulararch at a 25 percent stage in a plan view.

FIG. 59E. shows both arches at the 25 percent stage.

FIG. 59F shows the expected tooth position at the 50 percent stage.

FIG. 59G shows the position of both arches at the 50 percent stage.

FIG. 59H shows the position of expected position of the teeth at thetarget position (100 percent).

FIG. 59I shows the position of both arches at the 100 percent (finaltooth position).

FIG. 60 shows a representation of the tooth models using a “transparent”function in which the tooth is rendered in a semi-transparent manner,instead of being a solid object.

Presently preferred and alternative embodiments of the invention havebeen set forth. Variation from the preferred and alternative embodimentsmay be made without departure from the scope and spirit of thisinvention. Furthermore, the reference in the claims to an opticalscanner for scanning the dentition of the patient is intended toencompass both an in-vivo scanner scanning the teeth of the patientdirectly or the use of an optical, laser, destructive, or other type ofscanner scanning a physical model of the teeth of the patient or animpression thereof.

1-13. (canceled)
 14. In a workstation comprising a computing platformhaving a graphical user interface, a processor and a computer storagemedium containing digitized records pertaining to a patient, saiddigitized records including image data, and a set of softwareinstructions providing graphical user interface tools for access to saiddigitized records, a method for facilitating treatment planning for apatient comprising the steps of: providing the user with variables formanipulating a virtual model of the patient, said variables including atleast one of defining of occlusal planes, archforms, midline, referencetooth, tooth position in said virtual model, and space allocationbetween and among teeth in said virtual model; storing user inputs forsaid variables in said computer storage medium; providing the user withthe ability to change said variables using said graphical userinterface; retaining that the most current value for each such variablein said workstation and displaying said virtual model with said value byas said user is changing other of said variables during a treatmentplanning session.
 15. The method of claim 14, wherein said model of thevirtual patient comprises a three-dimensional model including softtissue, skeletal, and dental anatomical structures.
 16. The method ofclaim 14, wherein the treatment planning is performed with reference toconstraints for the patient, and wherein said constraints are identifiedvia user interface interaction with said virtual model of the patient.17-32. (canceled)
 33. In a workstation comprising a computing platformhaving a graphical user interface, a processor and a computer storagemedium containing digitized records pertaining to a patient, saiddigitized records including image data, and a set of softwareinstructions providing graphical user interface tools for access to saiddigitized records, the workstation further comprising softwareinstructions for facilitating treatment planning for a patient, saidinstructions comprising: instructions providing the user with variablesfor manipulating a virtual model of the patient, said variablesincluding at least one of markings of occlusal planes, tooth position insaid virtual model, and space allocation between and among teeth in saidvirtual model; instructions for receiving user inputs for said variablesand storing said user inputs in said computer storage medium;instructions for providing the user with the ability to change saidvariables using said graphical user interface; and instructions forretaining that the most current value for each such variable in saidworkstation and displaying said virtual model with said value by as saiduser is changing other of said variables during a treatment planningsession.
 34. The workstation of claim 33, wherein said model of thevirtual patient comprises a three-dimensional model including softtissue, skeletal, and dental anatomical structures.
 35. The workstationof claim 33, wherein the treatment planning is performed with referenceto constraints for the patient, and wherein said workstation comprisesinstructions for enabling a user to identify said constraints via userinterface interaction with said virtual model of the patient andentering said constraints into said computer storage medium. 36-46.(canceled)
 47. The method of claim 14, wherein the workstation providestools for manipulation of at least one of a) soft tissue, b) hardtissue, and c) functional elements. 48-57. (canceled)